Repository navigation
Conversation
Omit an unspecified profiler directory from EngineArgs kwargs so Config
can resolve its existing environment default. Explicit CLI values retain
precedence, and direct Config callers keep their current semantics.
Read an empty ATOM_TORCH_PROFILER_DIR as unset in envs, so `VAR=` clears
it instead of aborting startup on os.makedirs(""). Give the online-quant
dump its own ATOM_ONLINE_QUANT_INFO_DIR, falling back to the profiler
directory, and stop the ATOMesh launcher exporting a profiler directory
on every launch now that the engine honors it.
Cover the hand-off through EngineArgs.create_engine() -> LLMEngine ->
Config with the environment unset, empty and set, and with and without
an explicit --torch-profiler-dir.
Signed-off-by: Yaakoub El Khamra <176948988+yelkhamra@users.noreply.github.com>
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
valar-cop
left a comment
There was a problem hiding this comment.
some changes needed..
ci Offline inference smoke test failed — not sure yet (low): The captured log is empty (0 bytes), so the failing step cannot be identified. The job succeeded on main and on other recent runs, and this PR only changes arg_utils plus a new test, but that is not enough to attribute the failure.
next: Re-run the job and fetch its log, then classify it. https://github.com/ROCm/ATOM/actions/runs/41518846782/job/125529318605
| f.name: getattr(self, f.name) for f in fields(self) if f.name != "model" | ||
| } | ||
| # Let Config resolve ATOM_TORCH_PROFILER_DIR when no path was supplied. | ||
| if self.torch_profiler_dir is None: |
There was a problem hiding this comment.
empty ATOM_TORCH_PROFILER_DIR= now crashes startup, os.makedirs('').. read empty as unset in envs instead?
There was a problem hiding this comment.
Done as suggested: ATOM_TORCH_PROFILER_DIR now reads
os.getenv("ATOM_TORCH_PROFILER_DIR") or None in atom/utils/envs.py:771-774,
so ATOM_TORCH_PROFILER_DIR= is indistinguishable from unset and never reaches
Config as "". Covered by
tests/test_envs.py::TestEnvsOverrides::test_empty_torch_profiler_dir_reads_as_unset
and by test_torch_profiler_dir_handoff[-None-None], which drives the same
hand-off as the server. With the envs change reverted, that case fails with
FileNotFoundError: [Errno 2] No such file or directory: ''.
| } | ||
| # Let Config resolve ATOM_TORCH_PROFILER_DIR when no path was supplied. | ||
| if self.torch_profiler_dir is None: | ||
| kwargs.pop("torch_profiler_dir") |
There was a problem hiding this comment.
the workaround comment in pd_server_atom.sh is stale now, and it exports a dir on every launch.. please sweep it here
There was a problem hiding this comment.
Swept. .github/scripts/atomesh/pd_server_atom.sh no longer exports
ATOM_TORCH_PROFILER_DIR (lines 350-362), and the stale comment is replaced at
1948-1951 with what the per-worker flag does now.
The export had a second reader: _save_online_quant_info wrote rank 0's
online_quant_info_*.json under ATOM_TORCH_PROFILER_DIR when set. Removing
the export alone would have moved those dumps out of the results tree, so the
dump got its own knob, ATOM_ONLINE_QUANT_INFO_DIR (atom/utils/envs.py:891-900,
atom/model_loader/loader.py:180-182), which falls back to
ATOM_TORCH_PROFILER_DIR and then the working directory. The launcher exports it
to the same ${RUN_DIR}/online_quant[/${ATOMESH_EXECUTION_PHASE}]/rank-N path as
before, so the artifacts do not move.
The profiler itself is unchanged: only ATOMESH_TORCH_PROFILE_DURATION > 0 puts
--torch-profiler-dir on a launch, one directory per worker, and that flag still
outranks the environment default.
| kwargs = _engine_kwargs(argv) | ||
|
|
||
| # Match LLMEngine's filtering and run Config's real directory validation. | ||
| config_fields = {f.name for f in fields(config_module.Config)} |
There was a problem hiding this comment.
this re-implements the Config filter, not the real hand-off.. can we drive LLMEngine path once and add the empty-env case?
There was a problem hiding this comment.
Reworked. tests/test_arg_utils_profiler.py now goes
CLI -> EngineArgs.add_cli_args/from_cli_args -> EngineArgs.create_engine()
-> LLMEngine -> Config (the call openai_server and the examples use) and
asserts engine.config.torch_profiler_dir, plus that Config created the
directory. The field.name filtering is gone. Cases: env unset / empty / set,
each with no CLI path and with one; the empty-environment case is included. The
explicit empty CLI string is still checked at the kwargs boundary, since it does
not select the environment default.
90ac6b1 to
37b90a6
Compare
|
@valar-cop thanks for the review. On the CI failure: Classified as infra, not caused by this diff. The job's only annotation is |
|
ci triage for ab725c8:
|
|
Read the production side in full (the 20 changed lines in
Two things, neither blocking. Not repeating valar-cop's three inline points — all look addressed. 1. if args.torch_profiler_dir is None:
args.torch_profiler_dir = "./profiler_traces"
engine_args = EngineArgs.from_cli_args(args)So on that path the precedence is CLI > → Change: 2. The new "empty reads as unset" rule has two siblings that don't follow it. In
→ Change: give both the same treatment ( Checked and found not to be issues, listing them so nobody re-derives them:
Not verified: no GPU run, so I did not confirm that traces actually land in the resolved directory — as the body says, that is outside what this change touches. |
Honor ATOM_TORCH_PROFILER_DIR in the offline profiling example, which replaced an omitted flag with its own default before EngineArgs saw it. Resolve the way the engine does -- flag, then environment, then the example's ./profiler_traces -- and name the source that supplied the directory in the notice. Read an empty ATOM_ENGRAM_CACHE_DIR as its default cache path rather than "", which Path resolves to the working directory, and an empty ATOM_RTL_TRACE_DIR as None, like ATOM_TORCH_PROFILER_DIR. State the rule once in _optional_path_env / _path_env, which the profiler, online-quant and ATOM_V4_CG_INSTR_DIR knobs now use as well. Cover the example's resolution over seven cases, including the notice, and the two sibling variables with default, override and empty-value tests. Signed-off-by: Yaakoub El Khamra <176948988+yelkhamra@users.noreply.github.com>
37b90a6 to
5247b8b
Compare
|
@yitingw1 Thanks for reading the production side and for the mutation table; both points 1. variable supplied it, and the previous "using default: ./profiler_traces" text only when neither is set. An explicitly empty flag counts as not given, like an empty variable: the example always profiles, so it never leaves ""for Config to reject the way EngineArgs still would.
2. The sibling variables follow the same rule. Tests: defaults at I left the debug-dump directories ( Your "checked and found not to be issues" list matches what I re-derived for Same limitation as you noted: no GPU run, so traces landing in the resolved |
valarLip
left a comment
There was a problem hiding this comment.
⚠️ Action required before merge (4)
The core fix is right and still needed on main (arg_utils.py:878-880, with main's own pd_server_atom.sh:1945 comment documenting the bug). valar-cop's three asks and yitingw1's two points are addressed at head.
- Rebase onto current main.
docs/environment_variables.mdconflicts with main (merge-tree), so heavy CI has not run on this code since 37b90a6. Re-run it after the rebase. - Keep the online-quant fallback in the loader, not in the env knob.
envs.py:932-935makesenvs.ATOM_ONLINE_QUANT_INFO_DIRreturnATOM_TORCH_PROFILER_DIR's value when the knob is unset, so the attribute's name does not match what it returns. Make the knob plain_optional_path_env("ATOM_ONLINE_QUANT_INFO_DIR")and writeenvs.ATOM_ONLINE_QUANT_INFO_DIR or envs.ATOM_TORCH_PROFILER_DIR or os.getcwd()atloader.py:182; then drop the restated chain in the loader comment. - Fix the
resolve_profiler_dirdocstring.profile_offline.py:51-56says this is "the precedence every other entry point uses", but an empty--torch-profiler-diris treated as unset only here.EngineArgsforwards""toConfig, which fails onos.makedirs(""), as your owntest_empty_cli_directory_is_not_treated_as_nonepins. Say that the empty-flag rule is specific to this example. - Trim the extras. The
ATOM_V4_CG_INSTR_DIRedit (envs.py:1009) changes nothing, so revert it. The example only needsargs.torch_profiler_dir or envs.ATOM_TORCH_PROFILER_DIR or DEFAULT_PROFILER_DIRplus the print, not a(dir, notice)tuple with a 9-line docstring.
Reviewed at 5247b8b. Already fixed: empty env reads as unset (valar-cop), launcher export swept (valar-cop), real LLMEngine hand-off test (valar-cop), profile_offline honors env and sibling dir knobs (yitingw1).
…ample ATOM_ONLINE_QUANT_INFO_DIR reported ATOM_TORCH_PROFILER_DIR's value when its own variable was unset, so the attribute did not name what it returned. Make it a plain _optional_path_env of its own variable and write the fallback once at its only call site, in the loader: the dump's directory, else the profiler directory, else the working directory. Pin that effective directory in a new loader test, and have the envs test assert the knob reads only its own variable. Reduce the offline profiling example to the resolution it needs: the flag, or ATOM_TORCH_PROFILER_DIR, or the example's own directory, then a print of the directory. The empty-flag rule is now a comment where it applies and states that it is specific to this example, since EngineArgs forwards "" to Config. Revert the ATOM_V4_CG_INSTR_DIR edit, which changed nothing: the knob already read an empty value as None. Rebased onto 004f342; docs/environment_variables.md resolves to main's Engram rows plus this branch's single added sentence. Signed-off-by: Yaakoub El Khamra <176948988+yelkhamra@users.noreply.github.com>
5247b8b to
d9dd55b
Compare
|
Asks 2–4 are in commit 1. Rebase onto current main. 2. Online-quant fallback back in the loader. 3. 4. Extras trimmed. |
valarLip
left a comment
There was a problem hiding this comment.
⚠️ Action required before merge (2)
The fix itself is done: all four r20 asks are addressed at head, and the change still belongs in ATOM. With ATOM_ONLINE_QUANT_INFO_DIR and ATOM_TORCH_PROFILER_DIR set, the knob now returns None when its own variable is unset or empty (probe). The fallback chain is written only in loader.py:180-182. ATOM_V4_CG_INSTR_DIR is back to main's line (envs.py:1043). The example is the single or chain at profile_offline.py:54-56, with a comment saying the empty-flag rule applies only to that script.
- Run heavy CI on the rebased head. Pre Checkin, which includes the non-GPU unit tests, and the offline smoke test pass at d9dd55b. The ATOM, vLLM and SGLang accuracy jobs were skipped because the PR has no label, so heavy CI has not run since 37b90a6. Add
ci:atom(orci:full, sincepd_server_atom.shchanged) and merge once they pass. The branch still merges cleanly into main 004f342. - Reuse
atom_without_aiterin the new loader test (nit).tests/test_online_quant_info_dir.py:10-29writes its own try/except ImportError fallback to the stubs.tests/aiter_stub.py:295 atom_without_aiter()already does this, andtests/model_ops/test_sp_attention_dispatch.py:15andtests/test_deepseek_v41_mono_hi_runner.py:28use it. Replace the helper withwith atom_without_aiter() as load: return load("atom.model_loader.loader").
Re-reviewed at d9dd55b. Already fixed: rebase / docs conflict (r20 #1, CI still pending), online-quant fallback moved to loader (r20 #2), example comment scoped to the example (r20 #3), V4_CG_INSTR_DIR revert + example trimmed (r20 #4); valar-cop's three asks and yitingw1's two points still hold.
Replace the test's hand-written try/except ImportError fallback with aiter_stub.atom_without_aiter, the helper the other stand-in tests use, so the loader is loaded exactly as theirs is. That helper gated on `import aiter`, which succeeds on a host with an AITER build and no GPU: AITER defers its arch probe, and the submodule that binds a kernel -- reached from the loader's import chain -- then fails on the missing device. Take the stand-ins unless a GPU is available, the line tests/conftest.gpu_stack_import already draws. A GPU host and the CI unit runner (no AITER at all) keep the paths they had. Signed-off-by: Yaakoub El Khamra <176948988+yelkhamra@users.noreply.github.com>
|
The branch is still based on 1. Heavy CI on the rebased head. Nothing to change in code, and the label is already there: 2. Reuse Done exactly as suggested: @pytest.fixture(scope="module")
def loader():
with atom_without_aiter() as load:
return load("atom.model_loader.loader")
One adjunct was needed for that to load anything in a CPU image that has an Verified both ways, inside
The same image also goes from 8 failed + 189 errors to 279 passed / 12 skipped |
valar-cop
left a comment
There was a problem hiding this comment.
LGTM
ci Accuracy (Kimi-K3 DSpark DCP8 TP8) failed — not sure yet (low): The captured log is 0 bytes, so the failing step cannot be identified. The head commit ab725c8 only changes tests/test_online_quant_info_dir.py and the same workflow passed at d9dd55b, so nothing ties the failure to this diff, but that cannot be confirmed without the log. The cell runs through the vLLM plugin, whose Config passes torch_profiler_dir=None explicitly and so does not read the changed env default; its online_quant_config path lands in os.getcwd() both before and after the loader change.
.valar/ci/job_132683815462.log is 0 bytes — no log lines available
next: Re-run the job and fetch its log, then classify; no code change is expected from this PR. https://github.com/ROCm/ATOM/actions/runs/43671008159/job/132683815462
ci ATOM vLLM Test Gate failed — not sure yet (low): Aggregation gate (evaluate-test-gate over check-signal, download_aiter_wheel, atom-vllm-oot); its log is also empty, but the gate's red follows from the failed accuracy matrix job above. It should turn green once that job does.
.valar/ci/job_132700690021.log is 0 bytes — no log lines available
next: No separate fix; re-run after the Accuracy (Kimi-K3 DSpark DCP8 TP8) job is re-run and passes. https://github.com/ROCm/ATOM/actions/runs/43671008159/job/132700690021
valarLip
left a comment
There was a problem hiding this comment.
⚠️ Action required before merge (2)
Both r22 asks are done, and the change still belongs in ATOM. The loader test now uses with atom_without_aiter() as load (tests/test_online_quant_info_dir.py:10-20). The added GPU gate in tests/aiter_stub.py:305-315 leaves the CPU job (no AITER) and GPU hosts on the paths they took before. On a host with AITER and no GPU, the 12 files that use the helper went from 12 failed + 194 errors to 276 passed (2 unrelated topk_select failures). Heavy CI ran at ab725c8 with ci:full, and 23 of 24 accuracy cells pass.
- Re-run
Accuracy (Kimi-K3 DSpark DCP8 TP8)and merge once it andATOM vLLM Test Gateare green. Job 132683815462 failed with "vLLM server did not become ready in time" after 40×30 s. Weight loading had finished at about 03:48, and there is no traceback. The product code is identical to d9dd55b, where the same cell passed in 16.5 min (run 43660855824); the new commit only touches tests. So this looks like a startup timeout on the runner, but the gate is red until the re-run passes. Note thatci:fulldoes not run the ATOMesh model benchmark on PRs (atomesh-benchmark.yaml:510runs it only onworkflow_dispatch). Thepd_server_atom.shchange is covered only by thetest_atomesh_*unit tests, which pass. - Fix the
ATOM_RTL_TRACE_DIRcomment (nit).envs.py:897-899says "VAR=disables it". Nothing in Python readsenvs.ATOM_RTL_TRACE_DIR. The only consumer,.github/scripts/atom_test.sh:62,381, is gated byENABLE_RTL_PROFILER=1and reads${ATOM_RTL_TRACE_DIR:-/app/rtl_traces}, so an empty value falls back to/app/rtl_tracesand does not disable anything. Drop that sentence, or say the variable namesatom_test.sh's output directory underENABLE_RTL_PROFILER=1(credit: comment-analyzer).
Re-reviewed at ab725c8. Already fixed: loader test reuses atom_without_aiter (r22 #2); ci:full heavy CI ran at head (r22 #1, one cell pending re-run); r20 #1-#4, valar-cop's three asks and yitingw1's two points still hold. Branch merges cleanly into main 7a79a14.
The comment said an empty value disables tracing, but no Python code reads envs.ATOM_RTL_TRACE_DIR: the variable's only consumer is .github/scripts/atom_test.sh, which uses it as the output directory for `rtl trace -o` under ENABLE_RTL_PROFILER=1 and, there, an empty value falls back to /app/rtl_traces. Say that instead, and note the empty-reads-as-unset rule only for the Python reader, consistent with the other directory knobs. Signed-off-by: Yaakoub El Khamra <176948988+yelkhamra@users.noreply.github.com>
|
No rebase this round: the branch is still based on 1. Re-run We cannot re-run Actions jobs from here (no re-run rights on this repo), so 2. Fix the Done in |
valarLip
left a comment
There was a problem hiding this comment.
⚠️ Action required before merge (1)
The code is done. The change still belongs in ATOM, and I found nothing left to change at 4954cac. Only CI is outstanding.
- Merge once heavy CI is green at this head.
ATOM Test,ATOM vLLM TestandATOM SGLang Testfor 4954cac have been queued since 2026-11-16 15:03 UTC and have not started; 23 accuracy cells are pending. Pre Checkin, Benchmark artifact tests, Atomesh accuracy validation and the ATOMesh unit tests pass. The vLLM run includesAccuracy (Kimi-K3 DSpark DCP8 TP8), the cell that timed out at ab725c8, so it replaces the requested re-run. The product code is the same as at ab725c8 except theATOM_RTL_TRACE_DIRcomment, and the branch still merges cleanly into main 7762f29.
Re-reviewed at 4954cac (full diff). Already fixed: ATOM_RTL_TRACE_DIR comment now matches atom_test.sh:61-66,380-387 (r23 #2); loader test reuses atom_without_aiter (r22 #2); ATOM_ONLINE_QUANT_INFO_DIR reads only its own variable and the loader holds the fallback, profile_offline.py is a single or chain with the example-only empty-flag note, and the ATOM_V4_CG_INSTR_DIR edit is reverted (r20 #2-#4); rebased (r20 #1); valar-cop's three asks and yitingw1's two points still hold. The 136 PR tests pass locally; probes show empty ATOM_TORCH_PROFILER_DIR / ATOM_RTL_TRACE_DIR / ATOM_ONLINE_QUANT_INFO_DIR read as None and an empty ATOM_ENGRAM_CACHE_DIR reads as its default.
Fixes #2911
Motivation
Summary
Let an omitted
--torch-profiler-diruseATOM_TORCH_PROFILER_DIR, while preserving explicit CLI values.Rebased onto current main
004f342ab22dedbef6d476713eeabfe9dbdadcbb. Against that base, the environment-only case fails with the unmodifiedatom/model_engine/arg_utils.py, and the empty-environment case fails with the unmodifiedatom/utils/envs.py; both pass with this change. The rebase's only conflict wasdocs/environment_variables.md, where main had rewritten the Engram rows; the resolution keeps main's rows and wording and keeps this branch's single added sentence on theATOM_ENGRAM_CACHE_DIRrow. No rebase in the latest rounds: the base is still004f342ab, the branch merges cleanly into upstream mainac9664173, and the latest round adds one comment-only commit (4954cac80) on top of the test-onlyab725c857, itself on top ofd9dd55bc0.Review follow-up in the same pull request:
ATOM_TORCH_PROFILER_DIR=now reads as unset inatom/utils/envs.pyinstead of reachingConfigas""and aborting startup onos.makedirs("").ATOM_ONLINE_QUANT_INFO_DIRknob so its artifacts stay where they were.EngineArgs.create_engine()->LLMEngine->Confighand-off, with unset, empty and set environment values.--torch-profiler-dir, elseATOM_TORCH_PROFILER_DIR, else its own./profiler_traces. It always profiles, so the chain ends at a directory rather thanNone.ATOM_ENGRAM_CACHE_DIRreads as its default cache path instead of""(whichPathresolves to the working directory), and an emptyATOM_RTL_TRACE_DIRreads asNone, likeATOM_TORCH_PROFILER_DIR.ATOM_ONLINE_QUANT_INFO_DIRreports only its own variable and the fallback toATOM_TORCH_PROFILER_DIRthen the working directory is written once in the loader; the profiling example is a singleorchain plus a print (an empty flag reads as unset there because""is falsy, and the comment says that rule is example-specific); the no-opATOM_V4_CG_INSTR_DIRedit is reverted.aiter_stub.atom_without_aiter, like the other stand-in tests, instead of its owntry/except ImportErrorfallback. That helper keeps the real AITER kernels only where a GPU can run them: an AITER build on a GPU-less host imports, because its arch probe is deferred, and then fails inside that probe, so the stand-ins are taken there too.ATOM_RTL_TRACE_DIRcomment now describes its actual consumer instead of claiming an empty value disables tracing. No Python code consumes the setting;.github/scripts/atom_test.shuses it as thertl trace -ooutput directory underENABLE_RTL_PROFILER=1, and an empty value falls back to/app/rtl_tracesthere. The_optional_path_envreader stays, and the empty-reads-as-unset note is now scoped to Python, like the other directory knobs. Comment-only change; no behavior changes.Problem
On the base revision, a CLI launch with only
ATOM_TORCH_PROFILER_DIRset producesConfig.torch_profiler_dir=None, leaving the runner without a profiler output directory.From source reading: in this state the runner skips profiling because its profiler directory is
None, while/start_profileand/stop_profilestill return success, so no trace is written.Technical Details
Root cause
The
EngineArgsfield and CLI option default toNone;_get_engine_kwargs()forwards that value, andLLMEnginepasses it intoConfig, bypassing the field's environment-backeddefault_factory.Evidence
./profiler_traces, which is what the script did whether or not the variable was set.tests/test_arg_utils_profiler.py, the environment-only case fails against the unmodifiedatom/model_engine/arg_utils.pyand passes with this patch; the other cases pass in both runs.atom/utils/envs.pywithFileNotFoundError: [Errno 2] No such file or directory: ''and passes with the change.atom/examples/profile_offline.pyreplaced an omitted flag with./profiler_tracesbeforeEngineArgssaw it, so the environment variable could not apply there.tests/test_profile_offline.pydrivesmain()over seven flag/environment combinations; with the environment step removed, its two environment cases fail.envs.py,ATOM_ENGRAM_CACHE_DIR=reads as""andPath("") / "compressed_vocab_<hash>.npz"iscompressed_vocab_<hash>.npzin the working directory (atom/model_ops/engram/mapping.py:139).ATOM_ONLINE_QUANT_INFO_DIRused to returnATOM_TORCH_PROFILER_DIR's value when unset, so the attribute did not name what it returned. It now returns its own variable, andtests/test_online_quant_info_dir.pypins the effective directory at the loader's call site instead.Fix
atom/model_engine/arg_utils.py: remove only thetorch_profiler_dirkeyword when itsEngineArgsvalue isNone.atom/utils/envs.py: add_optional_path_env(empty ->None) and_path_env(empty -> named default), and read the directory knobs through them:ATOM_TORCH_PROFILER_DIR,ATOM_ONLINE_QUANT_INFO_DIRandATOM_RTL_TRACE_DIR->None;ATOM_ENGRAM_CACHE_DIR->~/.cache/atom/engram.ATOM_V4_CG_INSTR_DIRis unchanged: it already read an empty value asNone.atom/utils/envs.py,atom/model_loader/loader.py: addATOM_ONLINE_QUANT_INFO_DIRfor the rank-0online_quant_info_*.jsondump. The knob is its own variable only; the loader's single call site readsATOM_ONLINE_QUANT_INFO_DIR or ATOM_TORCH_PROFILER_DIR or os.getcwd().atom/examples/profile_offline.py: resolve the trace directory as--torch-profiler-dir or ATOM_TORCH_PROFILER_DIR or DEFAULT_PROFILER_DIR, store it beforeEngineArgsis built, and print it. An empty flag reads as unset here because""is falsy; that rule is specific to this example, sinceEngineArgsforwards""toConfigandos.makedirs("")aborts..github/scripts/atomesh/pd_server_atom.sh: stop exportingATOM_TORCH_PROFILER_DIRon every launch; exportATOM_ONLINE_QUANT_INFO_DIRto the same${RUN_DIR}/online_quant.../rank-Npath instead. The profiling window still passes a per-worker--torch-profiler-dir.tests/test_arg_utils_profiler.py,tests/test_profile_offline.py,tests/test_envs.py,tests/test_online_quant_info_dir.py: cover the hand-off, the example's resolution, the empty-value rule for the directory variables, and the online-quant dump's effective directory.tests/aiter_stub.py:atom_without_aitertakes the stand-ins unless a GPU is available. An AITER build on a GPU-less host imports, because its arch probe is deferred, and then fails inside that probe, so "AITER imports" is not "AITER can run here"; a GPU host and a runner without AITER behave as before.tests/test_online_quant_info_dir.pyloads the loader through it.Why it works
Omitting the keyword lets
Configevaluate its existing lazy environment default, giving precedence to an explicit CLI path, then the environment variable, thenNone. Reading the environment value withorkeeps an empty value from being mistaken for a directory. The example applies the same order before it buildsEngineArgs, so its own default applies only when neither the flag nor the variable is set. The online-quant dump keeps its historical home because its call site keeps the same fallback, without making the knob return another variable's value.Risk / compatibility
Default or explicit
Nonevalues supplied throughEngineArgsnow inherit the environment setting, includingConfig's existing directory creation and validation, per-rank directories, and prefill/decode directories when P/D disaggregation is enabled.An empty
ATOM_TORCH_PROFILER_DIR=now behaves exactly like an unset one. Before this change the CLI silently ignored it; without theenvs.pypart of the patch it aborts startup onos.makedirs("").Direct
ConfigandLLMEnginecalls supplyingtorch_profiler_dir=Noneretain their current semantics.With
--mark-tracealso set, the existing graph-capture profiler can now activate when the directory comes from the environment.The offline profiling example now follows the engine's order and ends at
./profiler_traces: with the variable set it writes where the variable points instead of always to its own default. An explicitly empty--torch-profiler-dircounts as not given there, like an empty environment value, because the example always needs a directory. It prints the directory it resolved instead of a per-source notice.The sibling directory variables change only when set to the empty string:
ATOM_ENGRAM_CACHE_DIR=now uses the documented cache path rather than the working directory, andATOM_RTL_TRACE_DIR=now reads as unset. The latter has no Python consumer today;.github/scripts/atom_test.shreads the shell variable itself, where${ATOM_RTL_TRACE_DIR:-/app/rtl_traces}already treated empty as unset. The debug-dump directories (ATOM_FWD_DUMP_DIR,ATOM_WEIGHT_DUMP_DIR,ATOM_DEBUG_TOPK_PATH) keep the empty string as their off switch and are unchanged.Plugin configuration still supplies explicit
Noneand is unchanged; plugin-mode profiling remains vLLM'sprofiler_config.The ATOMesh launcher gives the engine a profiler directory only while its profiling window is on. Its online-quant dump now names
ATOM_ONLINE_QUANT_INFO_DIR, but lands in the same directory as before, so collected artifacts do not move.Related #2112 proposes profiler-window controls; coordinate its argument/config changes with this environment-default fix.
Test Plan
GPU validation plan
None needed for this argument/default fix: the CPU regression reproduces the failure and verifies the resolved configuration.
GPU trace emission was not tested. Heavy CI with
ci:fullran atab725c857:23 of 24 accuracy cells pass at that head. The
Accuracy (Kimi-K3 DSpark DCP8 TP8)cell failed on a vLLM readiness timeout (40 x 30 s) after weight loadinghad finished, with no traceback; its product tree is identical to
d9dd55bc0,where the same cell passed, so a maintainer re-run is requested and merge is
gated on it and on
ATOM vLLM Test Gate.ci:fulldoes not run the ATOMeshmodel benchmark on PRs; the
pd_server_atom.shchange is covered by thetest_atomesh_*unit tests, which pass.CPU: the regression tests and lint commands are listed under Test Result.
Test Result
tests/test_arg_utils_profiler.pydrives the same hand-off as the server and the offline examples:EngineArgs.add_cli_args->EngineArgs.from_cli_args->EngineArgs.create_engine()->LLMEngine->Config, then assertsengine.config.torch_profiler_dirand thatConfigcreated the directory.It covers the environment unset, empty and set, each without and with
--torch-profiler-dir(six cases), plus an explicitly empty CLI string still being forwarded rather than replaced by the environment. OnlyConfig's checkpoint and quantization lookups and the engine's tokenizer andCoreManagerare mocked; argument forwarding,LLMEngine.__init__andConfiginitialization run for real.tests/test_profile_offline.pydrives the example'smain()up toEngineArgs.from_cli_argsover seven combinations: flag and variable absent, variable set, flag set, both set, empty variable, empty flag, and empty flag with the variable set. It pins the directory handed toEngineArgsand the printed directory.tests/test_envs.pycovers the empty-value rule forATOM_TORCH_PROFILER_DIR,ATOM_ENGRAM_CACHE_DIRandATOM_RTL_TRACE_DIR, and pins thatATOM_ONLINE_QUANT_INFO_DIRreads only its own variable (unset and empty, withATOM_TORCH_PROFILER_DIRset).tests/test_online_quant_info_dir.pycallsatom.model_loader.loader._save_online_quant_info, loaded throughaiter_stub.atom_without_aiter(the real kernels on a GPU host, the stand-ins where AITER is absent or cannot run), and pins the effective output directory: own knob, elseATOM_TORCH_PROFILER_DIR(unset and empty own knob), else the working directory; and that a rank other than 0 writes nothing.CPU environment: a local ATOM CPU container, Python 3.12.3, pytest 9.0.3.
With the patch: 136 passed.
The unit-test job's own condition is a runner with no AITER at all. This image has an editable AITER build, so that condition was reproduced by blocking AITER imports with a
sitecustomize.pyonPYTHONPATH:import aiterthen raisesModuleNotFoundError, andtests/test_online_quant_info_dir.pypasses (5 passed). On this image as it is, the same file passes through the new GPU check inatom_without_aiter; with the helper's oldimport aitergate it errors on all five cases inside AITER's arch probe. Both runs are inCHECKS.md, together with the twelve other files that use the helper going from 189 errors to 0 on this image.Unmodified
atom/model_engine/arg_utils.pyplus the new tests: only the environment-only case fails.Unmodified
atom/utils/envs.py: the empty-environment hand-off fails onos.makedirs(""), and the new env-var tests fail because the knob does not exist yet.The two sibling lambdas reverted to their pre-change spelling: exactly the two empty-value tests fail.
The example with its environment step removed: exactly the two environment cases fail.
The loader without the two fallback terms: exactly the two fallback cases fail.
envs.ATOM_ONLINE_QUANT_INFO_DIRrestored to the old chained lambda: exactly the knob tests fail.Related suites:
328 passed, 1 skipped.
Full native CPU suite, using the CI exclusions for plugin tests and GPU server integration, run on this branch and on a clean worktree of
004f342ab22dedbef6d476713eeabfe9dbdadcbb:The suite is not green in this CPU-only image for pre-existing reasons
(
torch.finfo()collection errors andNo gfx arch found). The branch's200-entry
FAILED/ERRORset is a strict subset of the base's 249: nothingfails on the branch that passes on the base. The 49 entries only the base has
are 45 in
tests/test_deepseek_v41_mono_hi_gate.pyand 4 intests/test_deepseek_v41_mono_hi_rope.py, both of which useatom_without_aiter; on this GPU-less image with an editable AITER build, thehelper's old
import aitergate handed them the real kernels, and AITER's archprobe failed, while the stand-ins load as they do on the unit-test runner. The branch also passes the 30 tests this PR
adds (11430 = 11351 + 49 + 30: 7 in
tests/test_arg_utils_profiler.py, 7 intests/test_profile_offline.py, 5 intests/test_online_quant_info_dir.pyand 11 in
tests/test_envs.py). Collection with the same flags shows only thatdelta, 12,934 tests against 12,904, with 76 collection errors on both.
Black and Ruff on the nine changed Python files (
atom/examples/profile_offline.py,atom/model_engine/arg_utils.py,atom/model_loader/loader.py,atom/utils/envs.py,tests/aiter_stub.py,tests/test_arg_utils_profiler.py,tests/test_envs.py,tests/test_profile_offline.py,tests/test_online_quant_info_dir.py):Latest round (
4954cac80, comment-only):tests/test_envs.pypasses (117 passed), the eight related suites used in the previous round still pass (328 passed, 1 skipped), thetest_atomesh_*unit tests pass (204 passed, 2 skipped), and Black and Ruff are clean onatom/utils/envs.py. The full native CPU suite reproduces the previous head's counts and its exact 200-entryFAILED/ERRORset (117 failed, 11430 passed, 1460 skipped, 83 errors). No test can distinguish a comment-only change, so the comment's claims were checked against its consumer instead: no Python module outside tests reads the variable, and both reads inatom_test.shfall back to/app/rtl_tracesfor an unset or empty value (seeCHECKS.md).Submission Checklist