Skip to content

SD3.5 Large and FLUX.1: forward guidance_scale, prompt_2 and negative_prompt_2 to the pipeline - #5557

Open
singhharsh1708 wants to merge 1 commit into
tenstorrent:mainfrom
singhharsh1708:fix/dit-runner-forward-guidance-prompt2
Open

singhharsh1708 wants to merge 1 commit into
tenstorrent:mainfrom
singhharsh1708:fix/dit-runner-forward-guidance-prompt2

Conversation

@singhharsh1708

Copy link
Copy Markdown

ImageGenerateRequest accepts guidance_scale, prompt_2 and negative_prompt_2, but TTDiTRunner.run (tt-media-server/tt_model_runners/dit_runners.py) passes only the prompt, negative prompt, steps and seed to run_single_prompt, so for SD3.5 Large and FLUX.1-dev the three fields never reach the pipeline: SD3.5's classifier-free guidance stays at the pipeline's 3.5, FLUX.1-dev's guidance embedding stays at 3.5, and the second text encoder gets the first prompt. SDXL honours the same fields through its own runner (#883, #927).

TTDiTRunner.run now adds the keyword arguments a _pipeline_kwargs(request) hook returns; the base returns none, and TTSD35Runner and TTFlux1Runner return guidance_scale, prompt_2 and negative_prompt_2. guidance_scale is forwarded only when the client sent it (model_fields_set), because the request default of 5.0 is SDXL's; a request without it keeps each pipeline's own default, so served output does not change for existing clients. FLUX.1-schnell accepts and ignores guidance, as before. Motif and QwenImage, whose pipelines take cfg_scale, forward nothing extra.

The receiving side is tenstorrent/tt-metal#65025, which adds the three arguments to PipelineAPIMixin.run_single_prompt. Against a tt-metal pin that predates it, the runner checks the run_single_prompt signature, logs a warning and drops the arguments it does not have, so a request that sets them still succeeds (today they are dropped silently).

New tests/test_dit_runner_request_fields.py, loading the real module the way test_video_stage_metrics_wiring.py does: explicit fields reach a recording pipeline for both runners, a default request leaves all three unset, Motif and QwenImage forward nothing, and a pipeline with the old signature gets the call without them plus one warning. On main 3 of the 7 fail; with this change tests/test_dit_runner_request_fields.py tests/test_video_stage_metrics_wiring.py tests/test_minimax_h3_lightx2v_runner.py tests/test_kontext_api.py tests/test_video_runner.py: 115 passed. Not run on device; no image-quality comparison was made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant