Repository navigation
Transformers 4.54 and 4.55 and gpt-oss - #52
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
IlyasMoutawwakil
changed the title
IlyasMoutawwakil
changed the title
IlyasMoutawwakil
changed the title
There was a problem hiding this comment.
just to accelerate and separate testing as I was seeing lots of failures in seq2seq (BaseModelOutput constraint)
| elif is_transformers_version(">=", "4.43.0") and isinstance(outputs.past_key_values, EncoderDecoderCache): | ||
| outputs.past_key_values = outputs.past_key_values.self_attention_cache.to_legacy_cache() |
There was a problem hiding this comment.
idk why but gpt bigcode is returning EncoderDecoderCache 🤦https://github.com/huggingface/transformers/blob/549ba5b8b69a595aa8c18b72165bf8ddf1c30c8e/src/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py#L488
will open a PR in transformers
There was a problem hiding this comment.
transformers PR in huggingface/transformers#40811
| "hiera": "hf-internal-testing/tiny-random-HieraForImageClassification", | ||
| "ibert": "hf-internal-testing/tiny-random-IBertModel", | ||
| "imagegpt": "hf-internal-testing/tiny-random-ImageGPTModel", | ||
| # "imagegpt": "hf-internal-testing/tiny-random-ImageGPTModel", # The mode is broken on transformers :/ |
There was a problem hiding this comment.
broken for a couple versions in transformers https://github.com/huggingface/transformers/blob/549ba5b8b69a595aa8c18b72165bf8ddf1c30c8e/src/transformers/models/imagegpt/modeling_imagegpt.py#L659
There was a problem hiding this comment.
transformers PR in huggingface/transformers#40811
| if hasattr(self.orig_forward, "__wrapped__"): | ||
| self.orig_forward = types.MethodType( | ||
| check_model_inputs_patched(self.orig_forward.__wrapped__), self._model | ||
| ) |
There was a problem hiding this comment.
we unwrap the faulty decorator and wrap with the fixed one
There was a problem hiding this comment.
transformers PR in huggingface/transformers#40811
There was a problem hiding this comment.
Pull Request Overview
This pull request updates optimum-onnxruntime to support Transformers 4.54 and 4.55, along with adding support for gpt-oss models. The changes focus on maintaining compatibility with both newer and older Transformers versions while adding support for the new gpt-oss architecture.
- Updates to support Transformers versions 4.54 and 4.55
- Adds support for the new gpt-oss model architecture
- Updates model testing utilities and configurations to accommodate new Transformers changes
Reviewed Changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Updates Transformers version constraint from <4.54.0 to <4.56.0 |
| tests/onnxruntime/testing_utils.py | Adds gpt_oss model entry and updates repository references |
| tests/onnxruntime/test_seq2seq.py | Moves seq2seq tests to dedicated file |
| tests/onnxruntime/test_modeling.py | Removes seq2seq tests moved to separate file |
| tests/onnxruntime/test_decoder.py | Adds gpt_oss support and updates cache handling |
| tests/exporters/onnx/utils_tests.py | Adds gpt_oss and updates model references |
| optimum/onnxruntime/modeling_seq2seq.py | Updates Whisper generation mixin and cache handling |
| optimum/onnxruntime/modeling_ort.py | Adds dtype parameter to _from_pretrained |
| optimum/onnxruntime/modeling_decoder.py | Updates cache handling for different model types |
| optimum/exporters/onnx/utils.py | Adds gpt_oss to supported models list |
| optimum/exporters/onnx/model_patcher.py | Updates patching for Transformers 4.54+ |
| optimum/exporters/onnx/model_configs.py | Adds GPTOssOnnxConfig and updates GPTBigCode |
| optimum/exporters/onnx/input_generators.py | Adds GPTBigCodeDummyPastKeyValuesGenerator |
| optimum/exporters/onnx/_traceable_decorator.py | New decorator for model inputs patching |
| .github/workflows/*.yml | CI workflow improvements and dependency updates |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
edbf568 to
2f8ad72
Compare
echarlaix
left a comment
echarlaix
left a comment
There was a problem hiding this comment.
Awesome work, thanks a lot @IlyasMoutawwakil !
|
|
||
| return wrapped_forward | ||
|
|
||
| if any(recordable_keys.values()): |
There was a problem hiding this comment.
not sure to understand why we need all of this, if you could add some comment to explain that would be awesome 🙏
There was a problem hiding this comment.
I just patched/fixed the function to accept use_cache as a positional argument and not fail with some models 😭
it's a new transformers decorator that was added around all forward passes to handle a lot of the use_cache, rtutn_tuple, output_attentions, .. arguments in one central place.
it's very hacky and breaks a couple things unfortunately but we can't remove it unfortunately as it now handles a big chunk of the "what should the model output and in what format" logic.
fixes #55 and #46