[zucchini-nlp]

What does this PR do?

As per title, we'll remove it anyway in the end so let's start raising warnings. I already ask all new models to not use Vision2Seq

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Cyrilvallez]

I don't remember all the discussions around this, but is ImageTextToText what will be fully future-proof? So we don't go with ForCausalLM in the end?

[zucchini-nlp]

I don't think we'll remove ImageTextToText in the near future though there were discussions on unifying "AutoForMultimodal" under one umbrella mapping. didn't yet start working on new auto class, but I think it will be a new auto-mapping that:

1. Unifies all generative model, either under one class or two classes (one for text and other for multimodality)
2. Main goal is to clean up garbage repetitive code and we don't plan to delete existing auto classes, otherwise it might break a lot of external libraries that depend on us. Until new auto is added, we'll have a surge of new VLMs added to the mapping which makes it harder to just remove it

Personally, I'm in favor of keeping ImageTextToText for a while as the recommended mapping

[Cyrilvallez]

LGTM!

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto