[chat] generate parameterization powered by GenerationConfig and UX-related changes
#38047
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the |
gante
changed the title
chat] generate parameterization powered by generation config and UX-friendly changeschat] generate parameterization powered by generation config and UX-related changes
gante
changed the title
chat] generate parameterization powered by generation config and UX-related changeschat] generate parameterization powered by GenerationConfig and UX-related changes
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
LysandreJik
approved these changes
May 12, 2025
Member
LysandreJik
left a comment
LysandreJik
left a comment
There was a problem hiding this comment.
Very cool! Played around with it locally, really like it 👌
| > [!TIP] | ||
| > You can also chat with a model directly from the command line. | ||
| > ```shell | ||
| > transformers chat --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct |
zucchini-nlp
pushed a commit
to zucchini-nlp/transformers
that referenced
this pull request
May 14, 2025
…UX-related changes (huggingface#38047) * accept arbitrary kwargs * move user commands to a separate fn * work with generation config files * rm cmmt * docs * base generate flag doc section * nits * nits * nits * no <br> * better basic args description
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The main goal of this PR is to enable user-friendly
generateparameterization. This also facilitates performance-related customization, which will be the focus of a follow-up PR.After the deprecation cycle, new users typing
transformers chat -hwill be redirected to a (new) docs intro section to generation arguments, instead of seeing a wall of CLI arguments. Fortransformerspower users,chatis now usable and can be parameterized without prior knowledge about the CLI. These changes were inspired by the idea that CLIs should be a conversation and that they shouldn't drown users in informationMore specifically, with this PR:
generateflag as a positional argument, present and future, as opposed to being limited to a set of hardcoded flags;generation_config.json, for power users to pass complexgeneratearguments that may be difficult to specify in a CLI;!!status, a new command, can be used to print state-related information, such as the currentgenerateflags!setcan now be used to set arbitrarygenerateflags!resetwas removed -- it was providing minimal benefits (relaunching the CLI with the previous command is the same) but it was requiring us to maintain and pass the input state around!stats-> not a valid command -> prints error and help)generateargs in the docs, allowing a soft-landing into the parameterization of the text generation universeExample usage: