Repository navigation
fix(frontend): reject over-context requests before streaming #12092
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
2142ea0
fix(frontend): preflight streaming backend errors
Aphoh 43f2ecd
fix(frontend): validate engine token budgets before streaming
Aphoh 8628dad
test(frontend): reconcile token budget request paths
Aphoh f183956
fix(sglang): handle engine-less encode registration
Aphoh d6c301f
fix(frontend): close token budget admission gaps
Aphoh 44741a9
fix(vllm): align frontend token budget validation
Aphoh 569598f
Merge origin/main into warnold/fix-streaming-context-overflow-status
Aphoh 5d7ad65
Merge remote-tracking branch 'origin/main' into warnold/fix-streaming…
Aphoh e649241
fix(frontend): encode backend token overflow policies
Aphoh 5f4e942
test(frontend): compare native token budget behavior
Aphoh bdd13b2
test(frontend): capture TRT streaming rejection
Aphoh f6b3589
refactor(frontend): simplify token budget contract
Aphoh f4fe58a
Merge remote-tracking branch 'origin/main' into warnold/fix-streaming…
Aphoh 8467eb6
Merge origin/main into warnold/fix-streaming-context-overflow-status
rmccorm4 df7cb7f
Merge branch 'main' into warnold/fix-streaming-context-overflow-status
rmccorm4 d9414b3
fix(trtllm): defer token budget when limit is inferred
Aphoh a63d228
Merge remote-tracking branch 'origin/warnold/fix-streaming-context-ov…
Aphoh 2c9d456
Merge branch 'main' into warnold/fix-streaming-context-overflow-status
rmccorm4 34d6e9a
fix(frontend): initialize pooling error metric type
Aphoh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.