Repository navigation
fix(acp): retry a failed session directory removal after the agent exits - #1626
Conversation
On Windows, teardown sent SIGTERM and removed the agent's working directory at once. Windows cannot remove a directory a live process holds as its working directory, so rmSync threw EPERM synchronously and took the worker down with WORKER_CRASH, losing the recording and forcing another model call on the next run. Wait for the child to exit before removing its directory, remove it at once when the child already exited, retry transient locks (maxRetries 10, retryDelay 200), and never let a failed removal throw: a leftover temp directory is harmless. Also notes in the app.command docs that on Windows executable must name a real executable file, since npm is npm.cmd. Fixes e2e-inc#1623 Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
There was a problem hiding this comment.
All reported issues were addressed across 5 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
okwasniewski
left a comment
There was a problem hiding this comment.
[claude-opus-5-5] automated review on behalf of @okwasniewski
Thanks. The fix is right: removing the directory after the agent exits is the root cause from #1623, and "removes the session directory only after the agent exits" fails on the old code. A few small asks.
Must fix
-
session.ts:166: a syncrmSyncwithmaxRetries: 10, retryDelay: 200blocks the event loop while it retries. With a removal forced to fail on Node 24, the worker froze for 9 s with 0 timer ticks. On Windows, a grandchild the agent started in the session cwd can still hold the directory after the direct child exits, becausechild.killonly terminates the direct child. That freeze then lands on every teardown, including mid-test ones. -
session.ts:158-176: the existingexitedpromise already settles onexitand on spawnerror, and is already settled when the child is gone. Theremovedflag, the already-exited branch and both listeners can collapse to one async line, which also fixes item 1:void exited.catch(() => rm(cwd, { recursive: true, force: true, maxRetries: 10, retryDelay: 200 })).catch(() => undefined);
with
rmfromnode:fs/promises. Tried locally: typecheck clean, executor tests 33/33. As it stands, thecloselistener, theexitlistener, the try/catch, theremovedguard and the retry options all survive mutation. -
Comments at
session.ts:158-160andexecutor.test.ts:500-503say the child holds the dir as its working directory.spawnatsession.ts:116sets nocwd; what holds it is whatever the agent starts there. Reword, or drop them along with item 2. -
Please drop the
docs/reference/config.mdx:509note from this PR.- It's a core runner doc in a
fix(acp)PR. docs/quickstart.mdx:43says to use WSL on Windows.- It contradicts the
'pnpm'and'npm'examples right next to it.
Native-Windows
app.commandis a separate decision. - It's a core runner doc in a
-
PR body: no CI workflow runs on Windows, so the EPERM path itself is unverified. Please say that instead of "covered by CI on Windows".
Nits
- Changeset: drop "with retries for transient locks, and a failed removal never throws". Users don't need the internals.
executor.test.ts:492: the comment implies Windows coverage. It checks ordering on POSIX only, because on Windows SIGTERM isn't catchable.- Optional: try removal at once and defer to
exitedonly on failure. That keeps POSIX from leaking the dir when the worker exits before a slow agent does.
This was an automated review.
Replace the sync rmSync retry loop with the async rm on the session exit promise: a removal forced to fail no longer blocks the event loop, and the removed flag, the already-exited branch, and both listeners collapse to one line. Skip the signal-based ordering test on Windows, where Node terminates the child immediately for SIGTERM, and drop the config docs note about app.command executables from this PR. Executor tests 33/33 pass; full acp suite 36/39 with the same 3 pre-existing cli.test.ts Chromium-as-root failures as the base head.
|
Review round addressed as 4dcf561. Must fix:
Nits:
Not taken: the optional try-removal-at-once. On POSIX Verified: executor tests 33/33 pass, full acp suite 36/39 with the same 3 pre-existing cli.test.ts Chromium-as-root failures as the base head; typecheck and oxlint clean. |
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
okwasniewski
left a comment
There was a problem hiding this comment.
[claude-opus-5-5] automated review on behalf of @okwasniewski
Re-review of 4dcf561. All CR 2 asks are done: the async rm runs on exited, the cwd comments are fixed, the config.mdx note is gone and the body is accurate. The teardown is correct on spawn failure, on a hung agent (the 3s SIGKILL path) and on an agent that already exited. CI is green.
Must fix
packages/acp/tests/executor.test.ts:496: the ordering test never waits onACP_EXIT_DELAY_MS. The assertions afterstep.end()are synchronous, so the test only catches a syncrmSync. An immediatevoid rm(cwd, ...)still passes, and so does dropping the env var or the fixture's delay handler inscripted-agent.ts:60-65. After theexistsSynccheck, add:This fails against an immediate asyncawait new Promise((resolve) => setTimeout(resolve, 500)); expect(() => process.kill(pid, 0)).not.toThrow(); expect(existsSync(cwd)).toBe(true);
rmand passes on your code. If you don't take it, drop the env var and the fixture handler, since they're dead code.
Should fix (maintainer call, not blocking)
2. session.ts:165: on POSIX, removal now waits for the agent to exit. After the last test, the worker exits through exitAfterFlush (run/worker/entry.ts:75). A real agent rarely exits that fast after SIGTERM, so each worker can leak one e2e-acp-* temp dir per run. On main, POSIX removed it right away. One option is to try rm at once and fall back to the exited path on failure, keeping the ordering test for that fallback. @okwasniewski: accept the leak or take the fallback?
Nits
session.ts:159-164: the comment can be two lines. "retry loop never blocks" is an internal detail.executor.test.ts:503-507describes Windows behavior in a test that runs only on POSIX.
Try removing the session directory at once, and fall back to removing it once the agent exits when that fails. On POSIX the removal usually succeeds even while the agent is still stopping, so a worker exiting right after the attempt leaves nothing behind; on Windows a process the agent started in the directory still holds it, so the removal fails there and waits for the exit. The ordering test now forces the immediate removal to fail, with a file in a read-only directory, so it keeps covering the fallback path, and waits 500ms after the step ends to catch an immediate async removal. The teardown comment is two lines now, and the test comment no longer describes Windows behavior in a POSIX-only test. Executor tests 33/33 pass as a non-root user (the CI environment); as root the read-only directory does not block removal, so the ordering test cannot demonstrate the fallback there. Full acp suite green with E2E_CHROMIUM_SANDBOX=0 for the real-Chromium cli tests. Typecheck clean, oxlint clean. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
|
Done in 3b6ec1c.
Nits: the session.ts comment is two lines now, and the test comment no longer describes Windows behavior in a POSIX-only test. Two notes: if you would rather accept the leak, the fallback is one line to revert. And the ordering test leans on file permissions, so as root the read-only directory does not block removal and the test cannot demonstrate the fallback there; CI runs non-root, where it passes. Executor tests 33/33 green as a non-root user. |
okwasniewski
left a comment
There was a problem hiding this comment.
Automated review.
Re-review of 3b6ec1c. Thanks for adding the at-once-with-fallback path. The code is correct: on Windows a failed first removal is caught and retried after the agent exits (no #1623 regression), and there are no unhandled rejections. What's left is wording plus one test.
Must fix:
- The wording no longer matches the behavior. On POSIX the immediate
rmsucceeds while the agent is still alive (checked withACP_EXIT_DELAY_MS=1500: 200ms afterendthe dir is gone and the pid is alive). So "removed only after the agent exits" is false in the changeset, the PR title, and the test name atexecutor.test.ts:495. Suggested changeset: "FixWORKER_CRASH (EPERM)on Windows when an ACP session ends: a failed session directory removal no longer crashes the worker and is retried after the agent exits." Rename the test to something like "retries the session directory removal after the agent exits when the first try fails". Retitle the PR to match. - The immediate path is untested. Reverting to remove-only-after-exit passes all 33 tests. Add a POSIX test with
ACP_EXIT_DELAY_MSset that checks the dir is gone while the pid is still alive. The test at:524passes under either version, so it can be repurposed for this (that also lets you drop the 50ms sleep at:535). - The fallback test fails as root, because the read-only dir doesn't block root. Add
|| process.getuid?.() === 0to itsskipIf.
Nits:
:511-512: the existence checks right afterstep.end()can't fail (the asyncrmhasn't run yet). Keep only the post-500ms checks.:507/:517: register achmodSync(cwd, 0o755)cleanup beforestep.end, so a failing test doesn't leave an undeletable dir in tmp.session.ts:159: one clause on why it tries at once (theexitAfterFlushleak).
Mutation results: 6/8 killed. The survivors are remove-only-after-exit (must fix 2) and the fallback without retries (Windows-only, expected).
The re-review found the wording no longer matched the behavior: on POSIX the immediate removal succeeds while the agent is still alive, so only after the agent exits was false in the changeset, the PR title, and the ordering test name. The changeset now says a failed removal is retried after the agent exits, the ordering test is renamed to say the same, and a new POSIX test checks the directory is gone while the agent is still alive (fails on the remove-only-after-exit code, passes with the fix). The fallback test now skips as root, where a read-only directory does not block removal, and restores the directory permissions on the way out so a failing run leaves no undeletable temp dir. Executor tests 32 passed, 1 skipped (the fallback test, as root); typecheck clean, oxlint clean. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
|
All three must-fix items from the re-review are done in 58e41bb:
Nits done too: the post-step.end() checks are dropped, the fallback test restores the directory permissions on the way out, and the teardown comment says why it tries at once (the worker can exit before the agent does). Executor tests 32 passed, 1 skipped (the fallback test, as root here); typecheck and oxlint clean. |
Automated review: CR 4 asks done in 58e41bb.
|
Automated review. Re-review of 58e41bb: all CR 4 asks are done and verified. Labeling Ready for Human Review.
Optional: the comment at For maintainers: there's no Windows CI, so the #1623 path is only covered by the chmod simulation. |
Fixes #1623.
On Windows, teardown() sent SIGTERM to the agent and then removed its session directory at once. Windows refuses to remove a directory while a process still has it as its working directory, so rmSync threw EPERM synchronously and took the worker down with WORKER_CRASH. Because the test never passed, no recording was saved, so every rerun burned another model call.
The directory is now removed at once, and retried after the agent exits when the first removal fails. On POSIX the first removal usually succeeds while the agent is still stopping, so a worker exiting right after the attempt leaves nothing behind; on Windows a process the agent started in the directory still holds it, so the removal fails there and waits for the exit. A failed removal never throws: a leftover temp directory is harmless.
Verified