Add Dia model #38405
Add Dia model #38405
Conversation
|
run-slow: dia |
|
This comment contains run-slow, running the specified jobs: models: ['models/dia'] |
src/transformers/modeling_utils.py
Outdated
|
|
||
| @abstractmethod | ||
| def encode(self, input_values: torch.Tensor, *args, **kwargs): | ||
| """Encode raw audio into discrete audio codebooks (with x channels)""" |
There was a problem hiding this comment.
i would just add whether we want it to be padded or not, batched or not, etc. It should support batched inputs, IDK for lists but say that output comes from a feature extractor maybe.
I think it should inherit from PreTrainedModel or should we say we have to inherite from DacPreTrainedModel + this?
Should be alright as well
There was a problem hiding this comment.
I think that's more part of the feature extractor / the interaction between model and feature extractor:
- Feature extractor outputs the input values (as batch or not)
- Relevant stuff like padding is also included there, e.g. padding via
hop_length - Additionally stuff like padding masks is also given if necessary but models seem to differ there if they need it
Added a bit more to the doc tho to ref to feature extraction; trying to change the inheritance, that's a good point!
| super().__init__(*args, **kwargs) | ||
|
|
||
| # Legacy behaviour just uses the tokenizer while new models use the processor as a whole | ||
| # at any given time | ||
| self.legacy = legacy |
There was a problem hiding this comment.
maybe something like self.no_processor or more descriptive will help users!
There was a problem hiding this comment.
let's add a compilation test as well!
There was a problem hiding this comment.
Discussed offline, will be moved to another PR - this is mostly for torch.export as base static + generate works with compile (and is tested through a common test)
|
run-slow: dia |
|
This comment contains run-slow, running the specified jobs: models: ['models/dia'] |
|
addressed the review comments, lmk if there's more @ArthurZucker will check slow runs on ci in a sec |
|
run-slow: dia |
|
This comment contains run-slow, running the specified jobs: models: ['models/dia'] |
|
run-slow: dia |
|
This comment contains run-slow, running the specified jobs: models: ['models/dia'] |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.