[SunMarc]

What does this PR do?

This PR fixes transformers serve as it is currently not working due to an unexpected field being passed request_id

reproducer

transformers serve
transformers chat localhost:8000 --model-name-or-path HuggingFaceTB/SmolLM3-3B

the second message will trigger an error.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[gante]

@SunMarc it's the other way around, the chat shouldn't pass the request ID (it's not a field in the API) 🤗

I thought this was already solved, a user opened a related PR a few days ago. Perhaps not 🤔

[SunMarc]

@SunMarc it's the other way around, the chat shouldn't pass the request ID (it's not a field in the API) 🤗

I thought this was already solved, a user opened a related PR a few days ago. Perhaps not 🤔

Yeah, it is not part of the create oai api but this is why we are passing it in extra_body + TransformersCompletionCreateParamsStreaming no so that we can pass it in generate_chat_completion just like generation_config no ?
In generate_chat_completion, we try to get the request_id also request_id = req.get("request_id", "req_0")

[gante]

@SunMarc that is logic that should be removed :p

The ID is something that is created by the server at generation time, mostly for internal server logging purposes. It is returned by the server (API), but not passed through a request.

[SunMarc]

Oh okay ! Then i'll just remove the request id from the extra_body then !

[SunMarc]

updated the logic in chat to not deal with request id ;)

[LysandreJik]

Ok!