Skip to content

Modify Processor trait to support chaining. - #1047

Merged
Narsil merged 1 commit into
clippy_updatefrom
modify_processor_trait
Aug 24, 2022
Merged

Narsil merged 1 commit into
clippy_updatefrom
modify_processor_trait

Conversation

@Narsil

@Narsil Narsil commented Aug 23, 2022 •

Copy link
Copy Markdown
Contributor

0 modifications yet, everything will consume the vector.
Every test should be green without any modifications.

I wanted to resurrect #1005 but I found myself having a real hard time making modifications and understanding what was what and why things were different when they were.

Instead I decided to start from scratch heavily inspiring from the PR instead.

This PR is the first step. It modifies the trait, and that' s it.

  • Next step making the various process_encodings actually agnostic (meaning not fusing their returned results into 1 encoding when they can).
  • Then we can add Sequence.

Each step will be in a different PR.

Edit companion PR for code linting : #1048

@Narsil
Narsil requested review from McPatate, SaulLu and mishig25 August 23, 2022 16:21
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

0 modifications yet, everything will consume the vector.
Every test should be green without any modifications.
@Narsil
Narsil force-pushed the modify_processor_trait branch from 8d16889 to c221192 Compare August 23, 2022 20:08
@Narsil
Narsil changed the base branch from main to clippy_update August 23, 2022 20:08
Comment thread tokenizers/src/processors/bert.rs Resolved

@mishig25 mishig25 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@Narsil
Narsil merged commit 1196e68 into clippy_update Aug 24, 2022
Narsil added a commit that referenced this pull request Aug 24, 2022
@SaulLu

SaulLu commented Aug 24, 2022

Copy link
Copy Markdown
Contributor

Thank you @Narsil 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants