Skip to content

test: support both HTTP clients in server integration tests - #4054

Merged
ananthsub merged 1 commit into
mainfrom
yuhez/server-test-client-compat
Oct 5, 2026
Merged

ananthsub merged 1 commit into
mainfrom
yuhez/server-test-client-compat

Conversation

@yuhezhang-ai

@yuhezhang-ai yuhezhang-ai commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Fresh server environments can select Starlette's httpx2-based TestClient while Gym's OpenAI SDK still requires httpx. This breaks the session tests when they inject an httpx.Cookies object into that client, and the GDPVal judge-routing test when it passes TestClient directly to the SDK. The failures also reproduce on unchanged main (b0e3fa6cf); they were observed in the full CI run for #3618.

Use a standard-library cookie jar in the two session test fixtures, preserving explicit-cookie session isolation. Forward the SDK's HTTP requests through an httpx.MockTransport into the real TestClient in the GDPVal test, preserving its proxy/model/credential assertions. Only test code changes; benchmark behavior and results are unaffected. A separate issue is unnecessary for this focused, reproduced CI repair.

Validation:

  • Before the fix on main with fresh server dependencies: 18 failed, 17 passed across the affected tests.
  • After the fix: 35 passed with fresh dependencies (Starlette 1.7.0, httpx2 2.13.1, httpx 0.28.1), and 35 passed in the locked development environment.
  • Commands: pytest resources_servers/example_session_state_mgmt/tests/test_app.py resources_servers/openenv/tests/test_app.py --import-mode=importlib -q; pytest resources_servers/gdpval/tests/test_judge_panel.py -q -k test_benchmark_panel_reaches_intended_upstream_models_and_credentials.
  • pre-commit run --all-files and git diff --check passed.
  • Model rollouts: N/A, test-harness compatibility only; no runtime environment/agent behavior changes.

The separate Aviary dependency failure from that CI run is addressed in #4055.

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yuhezhang-ai

Copy link
Copy Markdown
Contributor Author

/ok to test 0344cf1

@ananthsub
ananthsub added this pull request to the merge queue Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/37357656004

Merged via the queue into main with commit 060fbf0 Oct 5, 2026
38 checks passed
@ananthsub
ananthsub deleted the yuhez/server-test-client-compat branch October 5, 2026 18:53
ananthsub pushed a commit to ananthsub/Gym that referenced this pull request Oct 5, 2026
…Mo#4055)

An unconstrained `datasets` requirement lets a fresh Aviary environment
resolve `datasets==1.2.1` with current PyArrow. Importing the server
then fails with `AttributeError: module 'pyarrow' has no attribute
'PyExtensionType'`, before any Aviary tests run. This reproduces on main
(`b0e3fa6cf`) and appeared in [the full CI run for
NVIDIA-NeMo#3618](https://github.com/NVIDIA-NeMo/Gym/actions/runs/37337581691).

Require `datasets>=4.4.1`, matching the modern version already used by
Gym's locked development environment. A fresh server install now chooses
a compatible dataset stack instead of backtracking to the legacy
release. This is scoped to Aviary; a separate issue is unnecessary for
this focused, reproduced dependency repair.

Validation:
- Before: the standard server test failed during collection on unchanged
main.
- After, from a fresh server environment: `gym env test
--resources-server aviary +should_validate_data=true` passed (4 passed,
1 existing Docker-dependent test skipped); example-data validation
passed.
- The tests load the real GSM8K dataset and exercise environment
creation, a calculator tool step, and the dataset-server/client
lifecycle.
- `pre-commit run --all-files` and `git diff --check` passed.
- Model-generated rollouts: N/A for this dependency-compatibility
constraint; no agent, prompt, verifier, or scoring code changes. The
real dataset/tool lifecycle tests above ran; no model-generated answers
were tested.

The separate HTTP test-client failures are addressed in NVIDIA-NeMo#4054.

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
ananthsub added a commit that referenced this pull request Oct 6, 2026
## What

This PR only cherry-picks three already-merged `main` PRs into `r0.7.0`
to unblock failing CI on the release branch. It contains no new code.
Each commit applied cleanly with `git cherry-pick -x`, so each commit
message records its source commit.

| Original PR | Commit on `main` | Change |
|---|---|---|
| #4054 | `060fbf0d5` | test:
support both HTTP clients in server integration tests |
| #4055 | `10b38dbfd` |
fix(aviary): require a PyArrow-compatible datasets version |
| #4061 | `d85ae7930` |
fix(fabric): AUT-2479 constrain tokenizers to Python 3.13-compatible
releases |

## Why

`r0.7.0` CI fails on the same issues these PRs fixed on `main`. No
separate issue is needed: each original PR has its own context and
review.

## Validation

- All three commits cherry-picked onto `upstream/r0.7.0` without
conflicts.
- `ruff check` and `ruff format --check` pass on the changed test files.
- The original PRs were reviewed and validated on `main`. Release-branch
CI on this PR is the remaining check.

## Rollout evidence

N/A. These are test and dependency-pin changes cherry-picked unchanged
from `main`. They do not change environment or agent behavior.

---------

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Co-authored-by: Yuhe Zhang <yuhez@nvidia.com>
Co-authored-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>

This branch was previously deployed

1 inactive deployment
public — 0344cf1b Deployed Oct 5, 2026 by copy-pr-bot[bot] via release / finalize / notify #4323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants