Skip to content

fix(openai_model): bound refused provider connection retries - #5199

Merged
AdamRajfer merged 4 commits into
mainfrom
arajfer/fix-judge-connect-bound
Oct 29, 2026
Merged

AdamRajfer merged 4 commits into
mainfrom
arajfer/fix-judge-connect-bound

Conversation

@AdamRajfer

@AdamRajfer AdamRajfer commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

A provider that refused connections could leave the default openai_model client retrying indefinitely. Apply max_http_attempts to its connection retry budget as well, so the request eventually fails with its connection cause. Explicit single-attempt mode remains a single attempt.

Real loopback closed-port regressions fail on unchanged main (three failures and one passing single-attempt control) and pass with the fix: the default makes three attempts, a configured limit makes five, and the server returns HTTP 500 with ClientConnectorError. Native adjacent suites: 725 passed; all pre-commit hooks passed.

Additional fault validation used the combined #5072 + #5199 source 34b1a268b65c1356c9370664bc0e26feca637c64: the closed-judge path stopped after three connection attempts per call, the healthy-judge control completed 20/20, and the invalid-key control stopped after one judge attempt per call. The adapter implementation and tests match that source; its surrounding failure-reporting code comes from #5072. Saved attempt IDs and structured cause links therefore belong to that combined proof, rather than this PR alone.

The documentation follow-up explains the shared connection retry budget and that values of one or two lower request-timeout attempts from three to one or two; the default stays three. All repository pre-commit hooks passed. It changes no runtime code and required no new inference run.

Tested on main c155b77. This changes the provider client in openai_model; other model servers and readiness probes retain their policies.

Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
@AdamRajfer AdamRajfer added bug Something isn't working area:model Model servers, inference providers, and model adapters labels Oct 28, 2026
@AdamRajfer AdamRajfer self-assigned this Oct 28, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 28, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@yuhezhang-ai

Copy link
Copy Markdown
Contributor

Could you update the PR’s validation paragraph with the closed-judge, healthy-judge, and invalid-key results, identifying 34b1a26 as the combined #5072 + #5199 proof source?

The three changed files match this PR’s head, but the surrounding failure-reporting code differs, so the saved attempt IDs and structured cause links should be attributed to that combined source.

Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
@AdamRajfer

Copy link
Copy Markdown
Contributor Author

Updated the validation paragraph with the closed-judge, healthy-judge and invalid-key results, explicitly attributing the saved attempts and structured cause evidence to combined #5072 + #5199 proof source 34b1a26. The adapter implementation and tests match, while the surrounding failure-reporting code differs. No new inference run was performed.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

@svcnvidia-nemo-ci svcnvidia-nemo-ci added the needs-review PR is ready for code review and waiting on a reviewer label Oct 29, 2026

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHIP — LGTM — no concerns found.

Reviewed all four changed files and traced the adapter wiring through the shared client and transport retry branches. The connection limit preserves explicit single-attempt mode, and the regressions cover bounded refusal failures and HTTP 500 with the connection cause. Both documentation clarifications requested in the history are addressed.

Static inspection only; tests and documentation builds were not executed. The runtime topic was independently completed by the parent reviewer because its delegated terminal report was unavailable.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40880755061

Merged via the queue into main with commit f3db740 Oct 29, 2026
47 of 50 checks passed
@AdamRajfer
AdamRajfer deleted the arajfer/fix-judge-connect-bound branch October 29, 2026 13:54

This branch was previously deployed

1 inactive deployment
public — ed4e535d Deployed Oct 29, 2026 by copy-pr-bot[bot] via release / finalize / notify #8122
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:model Model servers, inference providers, and model adapters bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants