Skip to content

fix(provider): only send tool_stream for Z.AI providers in streaming path#5806

Merged
singlerider merged 2 commits intozeroclaw-labs:masterfrom
zavertiaev:fix/tool-stream-non-zai-providers
Apr 17, 2026
Merged

fix(provider): only send tool_stream for Z.AI providers in streaming path#5806
singlerider merged 2 commits intozeroclaw-labs:masterfrom
zavertiaev:fix/tool-stream-non-zai-providers

Conversation

@zavertiaev
Copy link
Copy Markdown
Contributor

Summary

  • Base branch target: master
  • Problem: stream_chat_with_tools unconditionally set tool_stream: true for all providers when streaming was enabled, causing 400 Bad Request on non-Z.AI OpenAI-compatible endpoints (e.g. litellm proxies) that reject the unrecognized tool_stream key.
  • Why it matters: Any user with a custom OpenAI-compatible provider (litellm, vLLM, etc.) cannot use streaming tool calls β€” every request fails with 400.
  • What changed: Streaming payload now uses self.tool_stream_for_tools() (same guard as non-streaming path) so tool_stream is only serialized for Z.AI providers.
  • What did not change (scope boundary): Non-streaming paths, Z.AI provider behavior, request structure for any other fields.

Label Snapshot (required)

  • Risk label: sk: low
  • Size label: size: XS
  • Scope labels: provider
  • Module labels: provider: compatible
  • Contributor tier label: (auto-managed)
  • If any auto-label is incorrect: N/A

Change Metadata

  • Change type: bug
  • Primary scope: provider

Linked Issue

Supersede Attribution (required when Supersedes is used)

N/A

Validation Evidence (required)

Commands and result summary:
```bash
cargo fmt --all -- --check # exit 0, no output
cargo clippy --all-targets -- -D warnings # exit 0, Finished dev profile
cargo test -p zeroclaw-providers # 762 passed; 0 failed; 0 ignored
```

  • Evidence provided: local test run output
  • If any command is intentionally skipped: N/A

Security Impact (required)

  • New permissions/capabilities? No
  • New external network calls? No
  • Secrets/tokens handling changed? No
  • File system access scope changed? No

Privacy and Data Hygiene (required)

  • Data-hygiene status: pass
  • Redaction/anonymization notes: N/A
  • Neutral wording confirmation: Yes

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

i18n Follow-Through (required when docs or user-facing wording changes)

  • i18n follow-through triggered? No

Human Verification (required)

  • Verified scenarios: Custom OpenAI-compatible provider (litellm proxy) previously returned 400 due to tool_stream key β€” confirmed root cause via docker logs.
  • Edge cases checked: Z.AI providers still get tool_stream: true (existing tests pass). Non-Z.AI providers with/without tools get None.
  • What was not verified: Live streaming with Z.AI provider (no access).

Side Effects / Blast Radius (required)

  • Affected subsystems: OpenAiCompatibleProvider::stream_chat_with_tools only
  • Potential unintended effects: None β€” Z.AI behavior preserved via existing requires_tool_stream() gate
  • Guardrails: Existing test suite (762 tests), plus new non_zai_provider_omits_tool_stream_regardless_of_streaming test

Agent Collaboration Notes (recommended)

  • Agent s used: Cursor AI
  • Workflow: Docker log analysis β†’ root cause in streaming path β†’ fix + test
  • Confirmation: naming + architecture boundaries followed (AGENTS.md)

Rollback Plan (required)

  • Fast rollback command: git revert <commit>
  • Feature flags or config toggles: None
  • Observable failure symptoms: 400 errors from non-Z.AI OpenAI-compatible proxies with Unrecognized key(s) in object: 'tool_stream'

Risks and Mitigations

  • Risk: Z.AI streaming could regress if requires_tool_stream() has a bug
    • Mitigation: Existing zai_tool_requests_enable_tool_stream and z_ai_host_enables_tool_stream_for_custom_profiles tests cover this

…path

The streaming method `stream_chat_with_tools` unconditionally set
`tool_stream: true` for all providers when streaming was enabled.
This caused 400 Bad Request errors on non-Z.AI OpenAI-compatible
endpoints (e.g. litellm proxies) that reject the unrecognized
`tool_stream` key.

The non-streaming path already correctly used `tool_stream_for_tools()`
which gates on `requires_tool_stream()` (Z.AI host or name check).
Apply the same guard to both streaming payload branches so the field
is only serialized for Z.AI providers.

Related: zeroclaw-labs#2901
@JordanTheJet JordanTheJet self-assigned this Apr 16, 2026
@JordanTheJet
Copy link
Copy Markdown
Collaborator

Verdict: Needs author action (minor)

Thanks @zavertiaev β€” small, surgical fix that closes a real bug for non-Z.AI OpenAI-compatible proxies (litellm, vLLM, etc.). Nice work.

Comprehension summary

  • What: In OpenAiCompatibleProvider::stream_chat_with_tools, the streaming request builder unconditionally set tool_stream: Some(true) whenever streaming was enabled. This PR routes both branches (tools and no-tools) through the existing self.tool_stream_for_tools(has_tools) helper, so tool_stream is only emitted for Z.AI-hosted providers (host api.z.ai / *.z.ai or provider name zai/z.ai), matching the non-streaming path.
  • Why: tool_stream is a Z.AI-specific request field. Non-Z.AI OpenAI-compatible endpoints (litellm proxies, vLLM, etc.) reject the unknown key with 400 Bad Request, breaking streaming tool calls for those users entirely.
  • Blast radius: Scoped to crates/zeroclaw-providers/src/compatible.rs streaming path. Z.AI providers retain the exact same behavior via requires_tool_stream() (unchanged). Non-Z.AI providers stop emitting a field they never should have emitted.

What I verified

  • Read the diff and confirmed both streaming branches now go through tool_stream_for_tools(...), aligning with the non-streaming paths at lines 1964 and 2062–2063 that already use the same helper.
  • Confirmed requires_tool_stream() at line 429 correctly gates on host (api.z.ai / *.z.ai) and provider name (zai/z.ai) β€” the Z.AI detection is robust and unchanged.
  • tool_stream_for_tools(false) returns None unconditionally (short-circuits on !has_tools), so the no-tools branch now never emits the field regardless of provider β€” desirable since tool_stream has no meaning without tools.
  • Only this one file in the repo references tool_stream β€” no downstream callers to regress.

Validation battery

  • cargo fmt --all -- --check β€” pass
  • cargo clippy --all-targets -- -D warnings β€” pass, zero warnings
  • cargo build β€” pass, zero warnings
  • cargo test --workspace β€” PR-relevant tests pass:
    • non_zai_provider_omits_tool_stream_regardless_of_streaming (new) β€” pass
    • non_zai_tool_requests_omit_tool_stream β€” pass
    • z_ai_host_enables_tool_stream_for_custom_profiles β€” pass
    • zai_tool_requests_enable_tool_stream β€” pass
  • Pre-existing test failures observed, NOT caused by this PR: Four flatten_system_messages_* tests in zeroclaw-providers fail on both master (origin/master at 9ca90b1b) and this branch. Root cause: tests call flatten_system_messages(&input, false) but expect merge behavior; the function's first line if !merge { return messages.to_vec(); } returns early. Introduced in commit d6dca4b8 (unrelated contributor). CI green because cargo nextest run --locked as invoked in .github/workflows/ci-run.yml apparently does not pick up these workspace-crate lib tests. Flagging as a separate CI-gap / test-calibration issue for maintainers β€” out of scope for this PR.

Security / performance assessment

  • Security: No impact. Removing a provider-specific payload key from providers that reject it does not expand attack surface or weaken deny-by-default. No auth, secrets, or validation paths touched.
  • Performance: No impact. One method call per request instead of a literal; allocation profile unchanged.

Architectural notes

  • Aligns streaming path with non-streaming paths (lines 1964, 2062–2063) that already use tool_stream_for_tools(...). Reduces divergence, not adds it.
  • No new dependencies, no binary-size impact, no runtime deps.

Findings

  1. [suggestion] Label snapshot typo / missing labels. The PR body lists Risk label: sk: low (should be risk: low) and Size label: size: XS, but the PR has zero labels applied on GitHub. This appears to be auto-labeling drift rather than a deliberate choice. Could you either apply risk: low, size: XS, provider, provider: compatible manually, or leave a note confirming auto-label will fire on next push?

What must change before re-review

  • Address the label suggestion above (or confirm it's intentional/auto-managed).

Once the labels land this is ready for maintainer merge. The code change itself is correct, minimal, well-tested, and aligned with the existing pattern.

@singlerider singlerider added risk: low Auto risk: docs/chore-only paths. size: XS Auto size: <=80 non-doc changed lines. bug Something isn't working labels Apr 16, 2026
Copy link
Copy Markdown
Collaborator

@singlerider singlerider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code is correct and well-tested. Approving.

@singlerider
Copy link
Copy Markdown
Collaborator

This will be merged after v0.7.0 is cut.

@singlerider singlerider merged commit 1da35db into zeroclaw-labs:master Apr 17, 2026
20 checks passed
whtiehack added a commit to whtiehack/zeroclaw that referenced this pull request Apr 18, 2026
…treaming paths

- stream_chat_with_tools δΈ€δΈͺεˆ†ζ”―ζ— ζ‘δ»Άε‘ tool_stream: Some(true),非 Z.AI
  provider ζŠ₯ 400
- δΈ€ε€„ιƒ½ζ”Ήζˆ tool_stream_for_tools() ι—¨ζŽ§οΌŒδΈŽιžζ΅εΌθ·―εΎ„δΈ€θ‡΄
- 加 non_zai_provider_omits_tool_stream ε›žε½’ζ΅‹θ―•

Ports upstream 1da35db (zeroclaw-labs#5806) to master_wecom's
pre-workspace-split layout.

Co-authored-by: Eugene Zavertiaev <152089142+zavertiaev@users.noreply.github.com>
WareWolf-MoonWall added a commit to WareWolf-MoonWall/zeroclaw that referenced this pull request Apr 18, 2026
… changelog

- Bump workspace version 0.7.0 β†’ 0.7.1 in root Cargo.toml
- Revert release workflow to gh release create --target for workflow_dispatch
  (the git-push approach from zeroclaw-labs#5860 is blocked by the org Restrict creations
  rule; --target uses the Releases API which bypasses it, and v0.7.1 has no
  immutable release lock so the previous blocker does not apply)
- Update CHANGELOG-next.md: retitle to v0.6.9 β†’ v0.7.1, restore full
  comprehensive notes from the upstream draft, and add entries that were
  missing from the original v0.7.0 draft:
    - feat(observability): otel_headers for authenticated OTLP exporters (zeroclaw-labs#5700)
    - feat: GitHub Copilot provider onboarding (zeroclaw-labs#5321)
    - fix(channels/telegram): inline_keyboard for tool approval requests (zeroclaw-labs#5790)
    - fix(provider): strip tool_stream for non-Z.AI providers (zeroclaw-labs#5806)
    - fix(agent): normalize empty successful tool output (zeroclaw-labs#5565)
    - fix(web): theme mode switch not applying correctly (zeroclaw-labs#5724)
    - fix(web): add visual preview swatches to theme selector (zeroclaw-labs#5767)
    - fix: cron_run tool output not delivered to configured channels
WareWolf-MoonWall added a commit to WareWolf-MoonWall/zeroclaw that referenced this pull request Apr 18, 2026
… changelog

- Bump workspace version 0.7.0 β†’ 0.7.1 in root Cargo.toml
- Revert release workflow to gh release create --target for workflow_dispatch
  (the git-push approach from zeroclaw-labs#5860 is blocked by the org Restrict creations
  rule; --target uses the Releases API which bypasses it, and v0.7.1 has no
  immutable release lock so the previous blocker does not apply)
- Update CHANGELOG-next.md: retitle to v0.6.9 β†’ v0.7.1, restore full
  comprehensive notes from the upstream draft, and add entries that were
  missing from the original v0.7.0 draft:
    - feat(observability): otel_headers for authenticated OTLP exporters (zeroclaw-labs#5700)
    - feat: GitHub Copilot provider onboarding (zeroclaw-labs#5321)
    - fix(channels/telegram): inline_keyboard for tool approval requests (zeroclaw-labs#5790)
    - fix(provider): strip tool_stream for non-Z.AI providers (zeroclaw-labs#5806)
    - fix(agent): normalize empty successful tool output (zeroclaw-labs#5565)
    - fix(web): theme mode switch not applying correctly (zeroclaw-labs#5724)
    - fix(web): add visual preview swatches to theme selector (zeroclaw-labs#5767)
    - fix: cron_run tool output not delivered to configured channels
WareWolf-MoonWall added a commit to WareWolf-MoonWall/zeroclaw that referenced this pull request Apr 18, 2026
Version
- Bump workspace version 0.7.0 β†’ 0.7.1 in root Cargo.toml + Cargo.lock

CI rationalisation (FND-004 Phase 1 β€” Rationalise)
- Delete checks-on-pr.yml and ci-run.yml β€” two workflows doing identical
  work on every PR, producing duplicate signal and double compute cost
- Add ci.yml (name: Quality Gate) β€” single staged pipeline replacing both:
    Stage 1: fmt + clippy --workspace (fast gate)
    Stage 2: build matrix, check all-features / no-default-features / 32-bit,
             benchmarks compile (parallel, gated on Stage 1)
    Stage 3: nextest (gated on Stage 1)
    Stage 4: cargo deny check β€” licenses, sources, advisories (deny.toml
             already present and triaged)
    Stage 5: CI Required Gate composite job (branch protection target)
- Remove rust_strict_delta_gate.sh β€” workspace-aware clippy --workspace
  makes delta comparison implicit (clean baseline = any warning fails)
- pre-release-validate.yml: remove pull_request trigger (secrets unavailable
  on fork PRs caused guaranteed failure on every Cargo.toml bump); remove
  stale CARGO_REGISTRY_TOKEN check (crates.io publishing removed in zeroclaw-labs#5858)

Release workflow
- Revert release-stable-manual.yml to gh release create --target for
  workflow_dispatch (git push approach from zeroclaw-labs#5860 blocked by org Restrict
  creations rule; Releases API bypasses it; v0.7.1 has no immutable lock)

Changelog
- Retitle CHANGELOG-next.md to v0.6.9 β†’ v0.7.1, restore full release notes,
  add entries missing from original draft: otel_headers (zeroclaw-labs#5700), GitHub
  Copilot onboarding (zeroclaw-labs#5321), Telegram inline_keyboard (zeroclaw-labs#5790), tool_stream
  fix (zeroclaw-labs#5806), empty tool output (zeroclaw-labs#5565), web theme fixes (zeroclaw-labs#5724, zeroclaw-labs#5767),
  cron_run delivery fix
WareWolf-MoonWall added a commit to WareWolf-MoonWall/zeroclaw that referenced this pull request Apr 18, 2026
Version
- Bump workspace version 0.7.0 β†’ 0.7.1 in root Cargo.toml + Cargo.lock

CI rationalisation (FND-004 Phase 1 β€” Rationalise)
- Delete checks-on-pr.yml and ci-run.yml β€” two workflows doing identical
  work on every PR, producing duplicate signal and double compute cost
- Add ci.yml (name: Quality Gate) β€” single staged pipeline replacing both:
    Stage 1: fmt + clippy --workspace (fast gate)
    Stage 2: build matrix, check all-features / no-default-features / 32-bit,
             benchmarks compile (parallel, gated on Stage 1)
    Stage 3: nextest (gated on Stage 1)
    Stage 4: cargo deny check β€” licenses, sources, advisories (deny.toml
             already present and triaged)
    Stage 5: CI Required Gate composite job (branch protection target)
- Remove rust_strict_delta_gate.sh β€” workspace-aware clippy --workspace
  makes delta comparison implicit (clean baseline = any warning fails)
- pre-release-validate.yml: remove pull_request trigger (secrets unavailable
  on fork PRs caused guaranteed failure on every Cargo.toml bump); remove
  stale CARGO_REGISTRY_TOKEN check (crates.io publishing removed in zeroclaw-labs#5858)

Release workflow
- Revert release-stable-manual.yml to gh release create --target for
  workflow_dispatch (git push approach from zeroclaw-labs#5860 blocked by org Restrict
  creations rule; Releases API bypasses it; v0.7.1 has no immutable lock)

Changelog
- Retitle CHANGELOG-next.md to v0.6.9 β†’ v0.7.1, restore full release notes,
  add entries missing from original draft: otel_headers (zeroclaw-labs#5700), GitHub
  Copilot onboarding (zeroclaw-labs#5321), Telegram inline_keyboard (zeroclaw-labs#5790), tool_stream
  fix (zeroclaw-labs#5806), empty tool output (zeroclaw-labs#5565), web theme fixes (zeroclaw-labs#5724, zeroclaw-labs#5767),
  cron_run delivery fix
theonlyhennygod pushed a commit that referenced this pull request Apr 18, 2026
… 1) (#5867)

Version
- Bump workspace version 0.7.0 β†’ 0.7.1 in root Cargo.toml + Cargo.lock

CI rationalisation (FND-004 Phase 1 β€” Rationalise)
- Delete checks-on-pr.yml and ci-run.yml β€” two workflows doing identical
  work on every PR, producing duplicate signal and double compute cost
- Add ci.yml (name: Quality Gate) β€” single staged pipeline replacing both:
    Stage 1: fmt + clippy --workspace (fast gate)
    Stage 2: build matrix, check all-features / no-default-features / 32-bit,
             benchmarks compile (parallel, gated on Stage 1)
    Stage 3: nextest (gated on Stage 1)
    Stage 4: cargo deny check β€” licenses, sources, advisories (deny.toml
             already present and triaged)
    Stage 5: CI Required Gate composite job (branch protection target)
- Remove rust_strict_delta_gate.sh β€” workspace-aware clippy --workspace
  makes delta comparison implicit (clean baseline = any warning fails)
- pre-release-validate.yml: remove pull_request trigger (secrets unavailable
  on fork PRs caused guaranteed failure on every Cargo.toml bump); remove
  stale CARGO_REGISTRY_TOKEN check (crates.io publishing removed in #5858)

Release workflow
- Revert release-stable-manual.yml to gh release create --target for
  workflow_dispatch (git push approach from #5860 blocked by org Restrict
  creations rule; Releases API bypasses it; v0.7.1 has no immutable lock)

Changelog
- Retitle CHANGELOG-next.md to v0.6.9 β†’ v0.7.1, restore full release notes,
  add entries missing from original draft: otel_headers (#5700), GitHub
  Copilot onboarding (#5321), Telegram inline_keyboard (#5790), tool_stream
  fix (#5806), empty tool output (#5565), web theme fixes (#5724, #5767),
  cron_run delivery fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working risk: low Auto risk: docs/chore-only paths. size: XS Auto size: <=80 non-doc changed lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants