Repository navigation
[TokenSpeed] Qwen3.8-Flash-Next FP4 GB300 AgentX update (upstream) / [TokenSpeed] Qwen3.8-Flash-Next FP4 GB300 AgentX 更新(上游分支) - #4922
Conversation
Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
tokenspeed-engine 0.1.0.post20261108, built from lightseekorg/tokenspeed@c5bb19e like the tokenspeed and kernel wheels, now published in lightseekorg/whl; pinned by SHA256. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
tokenspeed 0.1.1.post20261108 and CUDA 13 ARM64 kernel 0.1.4.post20261108, built from lightseekorg/tokenspeed@a5a298c, replace the release-candidate wheels. The engine wheel stays 0.1.0.post20261108 (built from c5bb19ef1); tokenspeed-engine/ is unchanged between the two revisions. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
Drop ple_embed_dtype from hf-overrides. TokenSpeed then stores the PLE n-gram table in BF16, its default; an A/B on the pinned 20261108 wheels measured every concurrency row within 1% of the FP8-table runs, with GSM8K 0.975. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
|
Thanks for the contribution!
中文感谢你的贡献!
|
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=42593696369 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=42593696369 |
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
Set chunked-prefill-size and max-prefill-tokens to 16384, the value in the current recipe, instead of 8192; prefill CUDA graphs stay capped at 8192 tokens. In a same-build A/B with 3600 s per row, 16K changed throughput by -0.2% to +2.8% and P90 interactivity by -1.4% to +7.4% per row, with the gains at concurrency 56 to 108, and lowered TTFT p90 at every row. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
|
/use 42593696369 |
|
@cquil11 staged run 42593696369. This run remains available across future |
…wen38-gb300-agentx-4915 # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
Ingest report — ✅ Results ingested into the Ingest workflow · Merge Ingest run · Benchmark source run
benchmarkFiles 10, rawBenchmarkRows 18, coveredByPerJobFiles 9, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 1319, gpuMetricSamplesNew 45309, powerPublicationPoints 9, changelogEntriesWritten 1 Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun. 中文Ingest 报告 — ✅ 结果已写入 上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。 |
Summary
Duplicates Fan Yang's submission #4915 onto a branch in
SemiAnalysisAI/InferenceXso the upstream full-sweep CI can run. The submission is copied from commit4a7629a04e31515baea30cad03ce44359e0bbdf1, preserving its commits and authorship.Updates
qwen3.8next-fp4-gb300-tokenspeed-agentic-mtpto the pinned TokenSpeed 20261108 wheels and FlashInfer 0.7.3 runner image, changes target/draft KV cache to FP8 E4M3, and introduces nine per-concurrency recipe variants. The original PR contains the full configuration rationale and contributor-reported pre-submission measurements; those measurements are not an InferenceX CI result.This PR uses
full-sweep-fail-fast. The original fork PR remains open. CI fixes will stay on this upstream branch; this task does not authorize merging either PR.Pins the runtime's
tachometer-scraperdownload tov2.66.1, alongside the CPU power exporter, instead of the movinglatestURL that repeatedly returned HTTP 504 during setup. The release has the same scraper asset hashes as the current latest release; upstream checksum verification remains enabled. This changes only dependency acquisition, not model precision, benchmark coverage, or evaluation thresholds. The runner-side fix and full serving/eval qualification still require a successful CI sweep.AI model disclosure
claude-opus-5-5[1m]: The original submission reports this model drafted the recipe bundle, master config, README, changelog, and description, and performed research and review.GPT-6 Astra: Duplicated the submission onto an upstream branch, prepared this description, monitored CI, and pinned the scraper download with a regression test.main.中文
将 Fan Yang 的 提交 #4915 复制到
SemiAnalysisAI/InferenceX仓库内的分支,以运行上游完整扫描 CI。来源提交为4a7629a04e31515baea30cad03ce44359e0bbdf1,保留原始提交和作者信息。更新
qwen3.8next-fp4-gb300-tokenspeed-agentic-mtp,使用固定版本的 TokenSpeed 20261108 wheels 和 FlashInfer 0.7.3 runner 镜像,将 target/draft KV cache 改为 FP8 E4M3,并提供九个按并发量划分的配置变体。原 PR 包含完整说明和贡献者报告的提交前测试结果;这些数据并非 InferenceX CI 结果。本 PR 使用
full-sweep-fail-fast。原 fork PR 保持打开,CI 修复仅提交到本上游分支,本次任务不包含合并授权。将运行时的
tachometer-scraper下载固定为v2.66.1,避免使用多次返回 HTTP 504 的latestURL。该版本与当前最新版本的 scraper 文件哈希一致,保留上游校验和验证,不改变模型精度、基准覆盖范围或评测阈值。运行节点上的修复效果及完整 serving/eval 验证仍需 CI 确认。GPT-6 Astra完成此下载修复和回归测试。AI 模型说明:原提交使用
claude-opus-5-5[1m];GPT-6 Astra负责复制分支、编写本说明和启动 CI 跟进。本次 changelog 冲突修复由 Codex 完成,当前会话的精确模型标识无法核实。Created with Perplexity