Skip to content

fix: fix fp8 hf_config_overrides bug - #2904

Merged
terrykong merged 6 commits into
mainfrom
ashors/fix-fp8-override-bug
Jul 9, 2026
Merged

terrykong merged 6 commits into
mainfrom
ashors/fix-fp8-override-bug

Conversation

@ashors1

@ashors1 ashors1 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

When hf_config_overrides and fp8 are both set, the user provided hf_config_overrides get overwritten with the fp8 hf_overrides. This PR adds to the existing hf_overrides if it exists to avoid overwriting the user-provided settings.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

Signed-off-by: Anna Shors <ashors@nvidia.com>
@ashors1
ashors1 requested a review from a team as a code owner June 23, 2026 20:02
@copy-pr-bot

copy-pr-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ashors1 ashors1 added the CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) label Jun 23, 2026
@ashors1

ashors1 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test a6979ab

@ashors1 ashors1 added CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) and removed CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) labels Jun 23, 2026
@ashors1

ashors1 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 6b3e868

@ashors1 ashors1 added the CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) label Jun 23, 2026
@ashors1

ashors1 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 3b81154

arnavk-nvidia
arnavk-nvidia previously approved these changes Jun 24, 2026

@terrykong terrykong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! The change is correct and well-scoped.

Context worth capturing: this bug was already fixed once and silently reverted, so it's effectively a re-fix rather than a new fix:

  1. #1413 (2025-10-25) added the original hf_overrides merge.
  2. #2188 "Add YaRN rope scaling support" (2026-04-04) removed those merge lines while refactoring the surrounding block, reintroducing the clobber.
  3. This PR (#2904) restores the merge.

Because there's no test covering fp8 + hf_config_overrides, the regression rode in unnoticed. Left an inline note suggesting a small regression test so this doesn't return a third time. Otherwise LGTM.

Generated by Claude Code

@ashors1
ashors1 requested a review from a team as a code owner July 7, 2026 19:09
@ashors1

ashors1 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 496146a

Signed-off-by: ashors1 <ashors@nvidia.com>
@ashors1

ashors1 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 2b0289a

@terrykong
terrykong enabled auto-merge (squash) July 9, 2026 18:31
@terrykong
terrykong merged commit 4f7b4aa into main Jul 9, 2026
137 of 139 checks passed
@terrykong
terrykong deleted the ashors/fix-fp8-override-bug branch July 9, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants