Skip to content

[TokenSpeed] Qwen3.8-Flash-Next FP4 GB300 AgentX update (upstream) / [TokenSpeed] Qwen3.8-Flash-Next FP4 GB300 AgentX 更新(上游分支) - #4922

Merged
cquil11 merged 11 commits into
mainfrom
upstream/tokenspeed-qwen38-gb300-agentx-4915
Nov 10, 2026
Merged

cquil11 merged 11 commits into
mainfrom
upstream/tokenspeed-qwen38-gb300-agentx-4915

Conversation

@functionstackx

@functionstackx functionstackx commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Duplicates Fan Yang's submission #4915 onto a branch in SemiAnalysisAI/InferenceX so the upstream full-sweep CI can run. The submission is copied from commit 4a7629a04e31515baea30cad03ce44359e0bbdf1, preserving its commits and authorship.

Updates qwen3.8next-fp4-gb300-tokenspeed-agentic-mtp to the pinned TokenSpeed 20261108 wheels and FlashInfer 0.7.3 runner image, changes target/draft KV cache to FP8 E4M3, and introduces nine per-concurrency recipe variants. The original PR contains the full configuration rationale and contributor-reported pre-submission measurements; those measurements are not an InferenceX CI result.

This PR uses full-sweep-fail-fast. The original fork PR remains open. CI fixes will stay on this upstream branch; this task does not authorize merging either PR.

Pins the runtime's tachometer-scraper download to v2.66.1, alongside the CPU power exporter, instead of the moving latest URL that repeatedly returned HTTP 504 during setup. The release has the same scraper asset hashes as the current latest release; upstream checksum verification remains enabled. This changes only dependency acquisition, not model precision, benchmark coverage, or evaluation thresholds. The runner-side fix and full serving/eval qualification still require a successful CI sweep.

AI model disclosure
  • claude-opus-5-5[1m]: The original submission reports this model drafted the recipe bundle, master config, README, changelog, and description, and performed research and review.
  • GPT-6 Astra: Duplicated the submission onto an upstream branch, prepared this description, monitored CI, and pinned the scraper download with a regression test.
  • Exact model identifier could not be verified for the Codex session that resolved this PR's changelog conflict with main.
中文

将 Fan Yang 的 提交 #4915 复制到 SemiAnalysisAI/InferenceX 仓库内的分支,以运行上游完整扫描 CI。来源提交为 4a7629a04e31515baea30cad03ce44359e0bbdf1,保留原始提交和作者信息。

更新 qwen3.8next-fp4-gb300-tokenspeed-agentic-mtp,使用固定版本的 TokenSpeed 20261108 wheels 和 FlashInfer 0.7.3 runner 镜像,将 target/draft KV cache 改为 FP8 E4M3,并提供九个按并发量划分的配置变体。原 PR 包含完整说明和贡献者报告的提交前测试结果;这些数据并非 InferenceX CI 结果。

本 PR 使用 full-sweep-fail-fast。原 fork PR 保持打开,CI 修复仅提交到本上游分支,本次任务不包含合并授权。

将运行时的 tachometer-scraper 下载固定为 v2.66.1,避免使用多次返回 HTTP 504 的 latest URL。该版本与当前最新版本的 scraper 文件哈希一致,保留上游校验和验证,不改变模型精度、基准覆盖范围或评测阈值。运行节点上的修复效果及完整 serving/eval 验证仍需 CI 确认。GPT-6 Astra 完成此下载修复和回归测试。

AI 模型说明:原提交使用 claude-opus-5-5[1m];GPT-6 Astra 负责复制分支、编写本说明和启动 CI 跟进。本次 changelog 冲突修复由 Codex 完成,当前会话的精确模型标识无法核实。

Created with Perplexity

Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
tokenspeed-engine 0.1.0.post20261108, built from lightseekorg/tokenspeed@c5bb19e like the tokenspeed and kernel wheels, now published in lightseekorg/whl; pinned by SHA256.

Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
tokenspeed 0.1.1.post20261108 and CUDA 13 ARM64 kernel 0.1.4.post20261108, built from lightseekorg/tokenspeed@a5a298c, replace the release-candidate wheels. The engine wheel stays 0.1.0.post20261108 (built from c5bb19ef1); tokenspeed-engine/ is unchanged between the two revisions.

Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
Drop ple_embed_dtype from hf-overrides. TokenSpeed then stores the PLE n-gram table in BF16, its default; an A/B on the pinned 20261108 wheels measured every concurrency row within 1% of the FP8-table runs, with GSM8K 0.975.

Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
@functionstackx
functionstackx requested a review from a team November 8, 2026 06:15
@functionstackx functionstackx added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +33 -1

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

Set chunked-prefill-size and max-prefill-tokens to 16384, the value in the current recipe, instead of 8192; prefill CUDA graphs stay capped at 8192 tokens. In a same-build A/B with 3600 s per row, 16K changed throughput by -0.2% to +2.8% and P90 interactivity by -1.4% to +7.4% per row, with the gains at concurrency 56 to 108, and lowered TTFT p90 at every row.

Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
@cquil11

cquil11 commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

/use 42593696369

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

@cquil11 staged run 42593696369.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

…wen38-gb300-agentx-4915

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@cquil11
cquil11 merged commit a72dd01 into main Nov 10, 2026
30 checks passed
@cquil11
cquil11 deleted the upstream/tokenspeed-qwen38-gb300-agentx-4915 branch November 10, 2026 03:50
@github-actions

Copy link
Copy Markdown
Contributor

Ingest report — ✅ Results ingested into the production database.

Ingest workflow · Merge Ingest run · Benchmark source run

Category new duplicate skipped unmapped failed
benchmark 9 0 9 0 0
eval 1 1 1 0 0
trace 9 0 0 0 0
power 9 0 0 0 0

benchmarkFiles 10, rawBenchmarkRows 18, coveredByPerJobFiles 9, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 1319, gpuMetricSamplesNew 45309, powerPublicationPoints 9, changelogEntriesWritten 1

Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun.

中文

Ingest 报告 — ✅ 结果已写入 production 数据库。

上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants