Repository navigation
Gemma3 hybrid fix - #42287
Gemma3 hybrid fix#42287
Conversation
|
run-slow: gemma3 |
|
This comment contains models: ["models/gemma3"] |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
CI ResultsModel CI Report❌ Failed tests
|
|
run-slow: gemma3 |
|
💔 This comment contains |
|
run-slow: gemma3 |
|
This comment contains models: ["models/gemma3"] |
CI Results✅ No failing test specific to this PR 🎉 ! |
|
|
||
| # cache_implementation="hybrid" an in the original transformers implementation | ||
| output = model.generate(**inputs, max_new_tokens=30, do_sample=False, cache_implementation="hybrid") | ||
| output = model.generate(**inputs, max_new_tokens=30, do_sample=False, cache_implementation="static") | ||
| output_text = self.processor.batch_decode(output, skip_special_tokens=True) | ||
|
|
||
| EXPECTED_NUM_IMAGES = 3 # one for the origin image and two crops of images |
There was a problem hiding this comment.
we don't need to replace "hybrid" with "static" cache, if we're not testing compile no?
There was a problem hiding this comment.
Talked about this with @Cyrilvallez , the choice we made was to put "dynamic" everywhere unless it did not match the expected result, in which case we used "static"
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gemma3 |
zucchini-nlp
left a comment
zucchini-nlp
left a comment
There was a problem hiding this comment.
Let's run slow tests and merge after it turns ✅
|
|
||
| # cache_implementation="hybrid" an in the original transformers implementation | ||
| output = model.generate(**inputs, max_new_tokens=30, do_sample=False, cache_implementation="hybrid") | ||
| output = model.generate(**inputs, max_new_tokens=30, do_sample=False, cache_implementation="static") | ||
| output_text = self.processor.batch_decode(output, skip_special_tokens=True) | ||
|
|
||
| EXPECTED_NUM_IMAGES = 3 # one for the origin image and two crops of images |
|
run-slow: gemma3 |
|
This comment contains models: ["models/gemma3"] |
CI Results✅ No failing test specific to this PR 🎉 ! |
* Fix gemma3 on H100 * Partial fixes for Mi325 * First half of A10 fix * Final A10 fix
In
gemma3modeling tests, there are still mentions of the deprecated cache typehybridThis PR removes those mentions and adjust the expectations when needed.