test(e2e): unblock Live Canary auth lanes after engine-v2 contract change

[ilblackdragon]

Summary

The Live Canary scheduled job (Auth Smoke, Auth Full, Auth Live Seeded) has failed every run for 3+ days, since the canary cut over to main on 2026-05-01. Three tests in test_v2_auth_oauth_matrix.py drive the failures, all rooted in the engine-v2 callable-only contract change from #2868 (merged 2026-04-25) that the tests pre-date.

What was broken

• test_mcp_same_server_multi_user_via_browser — engine v2 opens an approval pending_gate on the first MCP tool call. The browser fixture has no auto-approve UI, so the chat sat in pending_gate for the full 5-minute Playwright timeout. The expected_text_contains=\"Mock MCP search result\" predicate could never flip because the assistant bubble never received any content. (Failed in Auth Smoke + Auth Full.)
• test_wasm_tool_oauth_refresh_on_demand — same shape: the gmail call gates on approval before reaching the http credential-injection layer that triggers the OAuth refresh. Without approving, refresh_count never went above 0. (Failed in Auth Full.)
• test_wasm_tool_first_chat_auth_attempt_emits_auth_url — tests OLD engine v2 behavior (auto-emit gate_required Authentication on first call to an unauthed extension). After engine-v2: make available_actions callable-only for blocked providers #2868, the engine returns \"action 'gmail' is not callable in this execution context\" instead, and tool_activate(name=...) is the new model-facing enablement path. The canned mock LLM emits tool calls directly, so this scenario isn't reproducible from a scripted LLM. (Failed in Auth Full.)

Fix

• _wait_for_tool_call accepts a token= kwarg so multi-user tests can poll/approve as a per-user identity (backwards-compatible default).
• test_mcp_same_server_multi_user_via_browser — drive approval through the per-user API while polling for the tool call. Drop the broken predicate and tied text assertions; the bearer-token isolation assertion (what the test actually exists to verify) is retained.
• test_wasm_tool_oauth_refresh_on_demand — insert a _wait_for_tool_call approval step between _send_chat and _wait_for_refresh_request so the http credential layer actually runs.
• test_wasm_tool_first_chat_auth_attempt_emits_auth_url — marked xfail with an inline reason pointing at engine-v2: make available_actions callable-only for blocked providers #2868 and the replacement coverage (test_v2_tool_activate_surface.py, test_settings_first_gmail_auth_then_chat_runs).
• tests/e2e/conftest.py — set SECRETS_MASTER_KEY in ironclaw_server's spawned env. Without it, macOS local dev hangs on a Keychain authorization prompt during auto_generate_and_persist, so the fixture kills the process with SIGKILL after 60s and no browser-dependent test can run locally. CI Linux was unaffected (the keychain backend errors fast and the fallback writes to .env). Matches the pattern already used in auth_matrix_server, test_v2_engine_auth_cancel, test_v2_tool_activate_surface, etc.

The actual Auth Live Seeded failure (Timed out waiting for extension state: github) is a separate issue (real OAuth tokens / seeded credentials) and is not addressed here.

Verification

Reproduced each failure locally (HTTP-only repros for the non-browser tests, server-side log inspection of the multi-user test). Confirmed pending_gate=approval was the load-bearing symptom in both auto-callable scenarios.

$ pytest tests/e2e/scenarios/test_v2_auth_oauth_matrix.py::test_wasm_tool_oauth_refresh_on_demand
PASSED in 6.74s

$ pytest tests/e2e/scenarios/test_v2_auth_oauth_matrix.py::test_wasm_tool_first_chat_auth_attempt_emits_auth_url
XFAIL in 93s

$ pytest tests/e2e/scenarios/test_v2_auth_oauth_matrix.py -v --timeout=120
12 passed, 2 skipped, 3 xfailed (3 browser tests errored locally;
they'll run cleanly in CI)

Test plan

• Live Canary Auth Smoke and Auth Full jobs go green (next scheduled run after merge)
• Existing passing tests in test_v2_auth_oauth_matrix.py stay green

[gemini-code-assist]

Code Review

This pull request updates end-to-end tests to align with engine-v2 changes, primarily focusing on the new approval gating mechanism for tool calls. Key changes include adding a master key to the test environment, updating _wait_for_tool_call to support authentication tokens, and refactoring tests to explicitly handle tool approvals. Feedback suggests using asyncio.gather in test_mcp_same_server_multi_user_via_browser to poll and approve tool calls for multiple users concurrently, which would improve test performance.

[gemini-code-assist]

For a minor performance improvement in this test, you could run these two _wait_for_tool_call awaitables concurrently using asyncio.gather. This would allow both users' tool calls to be polled and approved in parallel, potentially reducing the test execution time.

Suggested change

	await _wait_for_tool_call(
	server["base_url"],
	owner_thread,
	"mock_mcp_mock_search",
	timeout=60.0,
	token=AUTH_TOKEN,
	)
	member_result = await send_chat_and_wait_for_terminal_message(
	member_page,
	"check mock mcp search",
	timeout=300000,
	expected_text_contains="Mock MCP search result",
	await _wait_for_tool_call(
	server["base_url"],
	member_thread,
	"mock_mcp_mock_search",
	timeout=60.0,
	token=member["token"],
	)
	await asyncio.gather(
	_wait_for_tool_call(
	server["base_url"],
	owner_thread,
	"mock_mcp_mock_search",
	timeout=60.0,
	token=AUTH_TOKEN,
	),
	_wait_for_tool_call(
	server["base_url"],
	member_thread,
	"mock_mcp_mock_search",
	timeout=60.0,
	token=member["token"],
	),
	)

[Copilot]

Pull request overview

Updates E2E auth/OAuth coverage to align with the engine-v2 callable-only tool contract (post-#2868) and unblocks Live Canary auth lanes by handling approval gating and deprecating an obsolete scenario.

Changes:

• Add per-user token support to _wait_for_tool_call and use API-driven approval to avoid browser fixtures hanging on pending_gate=approval.
• Mark an obsolete auth-on-first-call test as xfail under the new callable-only contract.
• Ensure SECRETS_MASTER_KEY is set for the spawned E2E server env and update the E2E workflow matrix to run the replacement v2 surface test.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
tests/e2e/scenarios/test_v2_auth_oauth_matrix.py	Adds per-user approval polling, updates multi-user MCP + Gmail refresh tests, and xfails an obsolete scenario.
tests/e2e/conftest.py	Sets SECRETS_MASTER_KEY in the spawned ironclaw_server env to prevent local macOS keychain prompts.
.github/workflows/e2e.yml	Replaces the removed/obsolete v2 auth preflight test with test_v2_tool_activate_surface.py in the matrix.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

tests/e2e/scenarios/test_v2_auth_oauth_matrix.py:1104

• _wait_for_tool_call currently POSTs /api/chat/approval for any pending_gate/pending_approval it sees. If the pending gate is something other than approval (e.g. gate_name == "authentication"), this will send an invalid approval request and fail with a confusing assertion. Consider checking pending.get("gate_name") == "approval" (or treating missing gate_name as approval) before auto-approving, and otherwise fail fast with a clear error that the tool call can’t proceed until the non-approval gate is resolved.

        pending = history.get("pending_gate") or history.get("pending_approval")
        if pending and pending["request_id"] not in approved_request_ids:
            approve = await api_post(
                base_url,
                "/api/chat/approval",
                token=token,
                json={
                    "request_id": pending["request_id"],
                    "action": "approve",
                    "thread_id": thread_id,
                },
                timeout=15,
            )
            assert approve.status_code == 202, approve.text
            approved_request_ids.add(pending["request_id"])

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[serrrfirat]

Approved to unblock merge queue after validating deterministic v2-engine E2E fixes. Full manual E2E workflow passed: https://github.com/nearai/ironclaw/actions/runs/25376351067

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[serrrfirat]

Re-approving after E2E helper-only follow-up for pending-message race; local web-regressions slice passed.