Skip to content

[PowerX] require Kimi-K3 B300 AgentX power / [PowerX] 强制校验 Kimi-K3 B300 AgentX 功耗 - #4488

Closed
edwingao28 wants to merge 1 commit into
mainfrom
fix/kimik3-b300-power-coverage
Closed

edwingao28 wants to merge 1 commit into
mainfrom
fix/kimik3-b300-power-coverage

Conversation

@edwingao28

@edwingao28 edwingao28 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Require valid power telemetry for Kimi-K3 B300 AgentX throughput; eval-only jobs remain exempt. Replaces #3632; tracks #3592.

Testing: Full sweep failed while Mooncake registered host memory. No measured point qualified.

Blocker: RDMA allocation failure remains unresolved.

AI model disclosure
  • Model/version: gpt-6-astra (recovery/review); original primary/delegated runtime identifiers unavailable.
  • Role: implementation, validation and integration.
PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.
中文

Kimi-K3 B300 AgentX 吞吐测量必须提供有效功耗遥测,纯评测任务保持豁免。替代 #3632,跟踪 #3592。

**验证:**完整 sweep 在 Mooncake 注册主机内存时失败,尚无合格测量点。

**阻塞:**RDMA 分配失败仍待解决。

改动类型:修复、配置与文档。gpt-6-astra 负责恢复和独立审阅;原始主 agent 与委派 agent 的完整运行时型号无法核实。各 agent 参与实现、验证与整合。

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@github-actions

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +25 -0

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

强制校验 Kimi-K3 B300 AgentX 的 GPU 功耗。
@edwingao28
edwingao28 force-pushed the fix/kimik3-b300-power-coverage branch from c1ead93 to c57c596 Compare October 25, 2026 08:08
@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

if benchmark.get(Fields.POWER.value):
entry[Fields.POWER.value] = True
if scenario.get(Fields.REQUIRE_POWER.value, scenario.get("require_power", False)):
entry[Fields.REQUIRE_POWER.value] = True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Kimi-K3 B300 AgentX runs started from e2e-tests.yml can still finish green with no valid GPU power, even though the scenario now sets require-power: true. The agentic throughput jobs there pass only ${{ inputs.require-power }} (e2e-tests.yml:399, e2e-tests.yml:465), so the matrix flag this line emits is never read. Fix: forward inputs.require-power || (matrix.config['require-power'] == true && matrix.config['eval-only'] != true) in both agentic jobs. That matches the multi-node job at e2e-tests.yml:354 and the five run-sweep.yml call sites. [also at: .github/workflows/run-sweep.yml:389 - Fork-PR sweeps and manual one-off runs of kimik3-fp4-b300-vllm-agentic-dspark can still pass with no validated GPU power.]

Why this was flagged

After this change, _agentic_entries at generate.py:1160-1161 adds require-power: True to every kimik3-fp4-b300-vllm-agentic-dspark row (nvidia-master.yaml:1516). run-sweep.yml reads that key at lines 389, 417, 447, 550 and 585. e2e-tests.yml does not. Its test-sweep-agentic job (line 399) and test-sweep-multi-node-agentic job (line 465) set require-power: ${{ inputs.require-power }} and never look at matrix.config['require-power']. When this config goes through e2e-tests.yml with the default require-power input, benchmark-tmpl.yml:100 and benchmark-multinode-tmpl.yml:146 get REQUIRE_POWER=0. Invalid telemetry is then recorded as power_valid: 0 and the job still passes. That is the failure the PR says it fixes. CONFIGS.md:96-97 says the flag requires power for throughput. The multi-node fixed-seq job in the same file (e2e-tests.yml:354) already ORs in the matrix flag, so these two agentic jobs are the only ones left out.

Verification: Trigger: the Kimi-K3 B300 agentic config runs through e2e-tests.yml. The test-sweep-agentic job (e2e-tests.yml:399) and the test-sweep-multi-node-agentic job (e2e-tests.yml:465) pass only require-power: ${{ inputs.require-power }}. The input defaults to false (e2e-tests.yml:33-37). On the base, e2e-tests.yml did not enforce power for this config either.

@github-actions

Copy link
Copy Markdown
Contributor

functionstackx commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

As mentioned last week to everyone,we will deprecate it and not accept any more merges the moment k3.1 pro weights drop. this means existing open PRs will not be accepted anymore after k3.1 pro weights drops.

The PR may not have been merged due to various factors:

  • accuracy concerns that weren't addressed before the k3.1 pro weights drop
  • msging core maintainers in slack for review only after the k3.1 pro weights dropped (in accordance with the agreed upon policy for review/merge SLO MODELS.md )
  • having patches
  • or other assortment of issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants