Skip to content

fix(responses): preserve refusal conversion semantics - #3189

Draft
ananthsub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/fix-response-refusal-edge-cases
Draft

ananthsub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/fix-response-refusal-edge-cases

Conversation

@ananthsub

@ananthsub ananthsub commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #3165 that closes the remaining refusal-conversion gaps.

  • Preserve an explicitly present empty refusal instead of converting it into empty text.
  • Reject Responses histories whose refusal/text ordering or multiple refusal parts cannot be represented losslessly by Chat Completions.
  • Declare native_finish_reason on Gym’s Chat Completion choice and Response models while omitting it from serialized output when absent.

@mahanfathi

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@yaoyu-33 yaoyu-33 added area:core Shared APIs, servers, telemetry, health, and registries bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer labels Sep 9, 2026
@github-actions github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Sep 10, 2026
@yaoyu-33

Copy link
Copy Markdown
Contributor

Could you clarify which upstream provider emits native_finish_reason independently of message.refusal, and which downstream evaluator consumes it? The PR description mentions BullshitBench, but the in-repo coverage currently demonstrates only passthrough. An integration test covering native_finish_reason="refusal" with no message.refusal would make the need for this public response extension and its expected contract clearer.

@yaoyu-33 yaoyu-33 added waiting-on-customer Waiting on the original author to respond complexity:low Localized change in one scope with a small, straightforward review surface and removed needs-review PR is ready for code review and waiting on a reviewer labels Sep 12, 2026
@yaoyu-33
yaoyu-33 marked this pull request as draft November 18, 2026 01:54
@yaoyu-33

Copy link
Copy Markdown
Contributor

We’re doing some cleanup of the open PR list. This PR has had no activity for over a month, so I’ve moved it back to draft.

If this change is still needed, please resolve any merge conflicts and mark the PR ready for review again when it’s ready to merge. Thanks!

@copy-pr-bot

copy-pr-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot removed the sla:review-overdue Review response is over the one-business-day SLA label Nov 18, 2026

This branch was previously deployed

1 inactive deployment
public — 835a3fb1 Deployed Sep 9, 2026 by copy-pr-bot[bot] via release / finalize / notify #2718
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Shared APIs, servers, telemetry, health, and registries bug Something isn't working complexity:low Localized change in one scope with a small, straightforward review surface waiting-on-customer Waiting on the original author to respond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants