Skip to content

feat(cli): e2e run --step holds each step until a key - #1540

Merged
okwasniewski merged 2 commits into
mainfrom
krzysztof/eng-1077-e2e-run-step
Nov 9, 2026
Merged

okwasniewski merged 2 commits into
mainfrom
krzysztof/eng-1077-e2e-run-step

Conversation

@KrzysztofMoch

@KrzysztofMoch KrzysztofMoch commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

e2e run had no way to stop between steps. --headed shows the app, but the test runs straight through. Playwright users reach for page.pause() or --debug and find nothing.

e2e run --step runs the first step, then holds each later top-level step until a key is pressed. The keys follow --watch.

Key Does
Enter runs the held step
n runs the rest of this test, holds at the next test
a runs the rest of the run without stopping
h shows or hides every key
q, Esc stops the run like Ctrl-C (exit 130)
 ❯  web  tests/coffee.e2e.ts 0/2
   └── orders two drinks paused
       ✓ app.open "/" 50ms
       ▶ step 2  locator.click "getByRole("radio", name: "Latte")"  tests/coffee.e2e.ts:7
         press enter to run this step, press h to show help, press q to quit

How it works

sequenceDiagram
  participant W as worker (StepRecorder)
  participant S as scheduler
  participant G as StepGate (CLI)
  W->>W: AttemptHolds stops the test clock
  W->>S: step-hold
  S->>G: beforeStep(step, { signal, show })
  G-->>S: resolves on Enter / n / a
  S->>W: step-release
  W->>W: clock runs again with the time it had left
Loading
  • RunOptions.beforeStep (packages/e2e/src/run/step-hold.ts) is an internal seam. It is not public API, so there are no .d.ts or wire schema changes. A worker holds each top-level step over IPC (step-hold, step-release, step-cancel).

  • Held time counts toward no timeout. AttemptHolds stops the test clock and rearms the deadline with what was left. The scheduler pushes maxDuration back by the held time.

  • Setup tests, retries, teardown steps, and steps inside an expect.poll read run without stopping.

  • Steps held together in one Promise.all share one prompt and run on one Enter. A waitForResponse placed beside the click that triggers it (the pattern in docs/browser.mdx) would otherwise run alone and time out while the click waits for its key.

  • The prompt is drawn in the list reporter's live window under its test. Without a live window, it is one plain line.

  • --step is refused, with exit 2:

    • in CI;
    • without a TTY;
    • with --watch;
    • with --workers above 1;
    • with the json reporter.

    It forces one worker.

  • Docs:

    • docs/debugging.mdx gets a "Step through a test" section.
    • The CLI reference, the errors reference, the Playwright migration rows (page.pause(), --debug), and the skill references are updated.
    • Patch changeset.

Issue

Closes #1251

Verified

Ran it locally: yes

  • Surface: the packed e2e build in a scratch project. It has a one-page app and two test files. Each run was driven in tmux on LBox with e2e run --step.
  • Coffee test, keys: Enter runs one step and moves to the next prompt. h shows the full key list and h again hides it. n finishes the test and holds at step 1 of the next test. a runs to the end. Result: 2 passed, exit 0.
  • Promise.all test, 7 s pause: both steps show in one prompt. I waited 7 s before pressing Enter, longer than the 5 s waitForURL gets. The test passed with exit 0.
   └── waits for the page a navigation opens paused
       ✓ app.open "/" 50ms
       ▶ step 2  browser.waitForURL "/\/menu$/"  tests/together.e2e.ts:6
       ▶ step 3  app.open "/menu"  tests/together.e2e.ts:6
         press enter to run these 2 steps, press h to show help, press q to quit
 ...
 ✓  web  tests/together.e2e.ts (1 test) 8.61s
  • Integration tests: tests/integration/step-hold.test.ts checks against real workers that:
    • a hold stops the test clock and the time limit;
    • a test that hangs after a release still times out on the time it had left;
    • an interrupt ends a held step as interrupted;
    • setup tests, teardown, retries, and poll reads are never held.
  • Unit tests: tests/unit/cli-step.test.ts checks the keys, the 150 ms guard, and the shared Promise.all prompt.
  • Headed run on macOS: Krzysztof ran an earlier build of the same demo with --step --headed on a Mac. That was before the Promise.all grouping change.

Reviewed with sol-loop: 1 round with gpt-6.1-sol. Sol verdict: approved. That approval was before the Promise.all grouping, which a later fresh-context review asked for.


Written by Claude Opus 5.5 in Claude Code (T3 Code).

View guided diff Turn on auto-fix

@github-actions github-actions Bot added the size:XXL 1,000+ changed lines, not counting tests, fixtures, translated docs, or the lockfile label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 91 passed

All 91 tests in 41 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 681ms
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 526ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 155ms
🟢 tests/async-states.e2e.ts · 2 passed 7.1s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.3s
🟢 async states › a wrong code is rejected after the toast has gone 4.8s
🟢 tests/canvas-only.e2e.ts · 2 passed 264ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 145ms
🟢 canvas only › a shape out of order paints the error and resets progress 119ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 385ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 234ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 151ms
🟢 tests/cart-totals.e2e.ts · 2 passed 342ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 211ms
🟢 cart totals › a quantity never drops below one 131ms
🟢 tests/checkout-review.e2e.ts · 1 passed 165ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 165ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 495ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 265ms
🟢 context menu trap › the decoy menu items refuse and keep the file 230ms
🟢 tests/control-inventory.e2e.ts · 8 passed 3.6s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 250ms
🟢 control inventory › check picks a radio the pick replaces with its summary 157ms
🟢 control inventory › a long press and a double click are told apart from a tap 974ms
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 235ms
🟢 control inventory › setInputFiles attaches files named from the project root 153ms
🟢 control inventory › the details view is left through the browser history 200ms
🟢 control inventory › scrollIntoView brings the footnote into view 247ms
🟢 control inventory › every exercise done shows the success message 1.4s
🟢 tests/css-content-ui.e2e.ts · 2 passed 301ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 160ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 141ms
🟢 tests/date-picker.e2e.ts · 2 passed 559ms
🟢 date picker › navigates three months ahead and books the required date 328ms
🟢 date picker › confirming without the exact date shows the error 231ms
🟢 tests/debounced-search.e2e.ts · 2 passed 2.8s
🟢 debounced search › a decoy one character off is refused 1.4s
🟢 debounced search › picking the exact item adds it to the cart 1.4s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.1s
🟢 one submit opens one confirm, even when double-clicked 1.1s
🟢 tests/div-soup.e2e.ts · 2 passed 614ms
🟢 div soup › a wrong code is rejected 277ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 337ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 611ms
🟢 drag and drop › an out-of-order item resets the dropzone 312ms
🟢 drag and drop › dropping Banana then Cherry passes 299ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 228ms
🟢 filter deep link › filtering writes the query to the URL 125ms
🟢 filter deep link › opening a shared link restores the filter 103ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 519ms
🟢 gift card purchase › refuses to buy without an amount 136ms
🟢 gift card purchase › refuses a recipient without an at sign 196ms
🟢 gift card purchase › sends the gift card 187ms
🟢 tests/hover-menu.e2e.ts · 2 passed 412ms
🟢 hover menu › a decoy leaf is refused 179ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 233ms
🟢 tests/iframe-form.e2e.ts · 2 passed 545ms
🟢 iframe form › a wrong code is refused by the host page 263ms
🟢 iframe form › the code read from one frame applies through the other 282ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 487ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 112ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 178ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 197ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 2.7s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 2.7s
🟢 tests/kanban-board.e2e.ts · 3 passed 700ms
🟢 kanban board › pointer drags move both cards into the goal layout 351ms
🟢 kanban board › a swipe along a path drags a card as well 165ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 184ms
🟢 tests/login-form.e2e.ts · 4 passed 716ms
🟢 login form › rejects a malformed email 183ms
🟢 login form › rejects a short password 165ms
🟢 login form › rejects an unknown account 166ms
🟢 login form › signs in and logs out 202ms
🟢 tests/lying-labels.e2e.ts · 4 passed 627ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 132ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 106ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 211ms
🟢 lying labels › visible text drives the payment through 178ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 416ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 137ms
🟢 native dialogs › dismissing the confirm cancels the order 132ms
🟢 native dialogs › a wrong gift code is rejected 147ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.2s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.2s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 4.2s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 235ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 316ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.1s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 1.3s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 1.4s
🟢 tests/order-history.e2e.ts · 2 passed 1.4s
🟢 order history › overview and settings render their panels 178ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.2s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.4s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 463ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 136ms
🟢 otp auto-advance › a wrong code clears every box and starts over 781ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 718ms
🟢 continues only after the banner and the overlay are gone 718ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 560ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 272ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 288ms
🟢 tests/price-sorting.e2e.ts · 3 passed 390ms
🟢 price sorting › opens in the featured order with sale prices struck through 106ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 127ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 157ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.1s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.1s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.3s
🟢 buys a deal once the promo chrome is out of the way 1.3s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 444ms
🟢 rich text editor › bolding exactly the required word passes the check 226ms
🟢 rich text editor › bold that spills into the previous word fails the check 218ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 548ms
🟢 saved addresses › adds an address and clears the form 186ms
🟢 saved addresses › ignores an add with an empty field 183ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 179ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 544ms
🟢 shadow DOM form › rejects a wrong access code 266ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 278ms
🟢 tests/smoke.e2e.ts · 1 passed 397ms
🟢 home lists the scenarios and opens one 397ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.1s
🟢 taps the moving target three times across rebuilds of the list 2.1s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 185ms
🟢 accepts the terms once the button clears the sticky footer 185ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 15.0s
🟢 scrolls the windowed table to the Golden Row and claims it 15.0s
🟢 tests/wishlist.e2e.ts · 2 passed 1.6s
🟢 wishlist › saving toggles the button and the tab badge 296ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.3s

Tested by e2e 0.22.0 · 1m 1s · web · run artifacts

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 87.7k tokens (85% cached) · $0.0053

All 33 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.4s
🟢 control inventory › check sets the checkbox and the radio 1 step 693ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 274ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 882ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.6s
🟢 control inventory › upload attaches the named project files 1 step 2.4s
🟢 control inventory › back returns from the details view to the inventory 1 step 279ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 338ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 354ms
🟢 act signs in with the benchmark account 1 step 354ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 13.2s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 6.7s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.9s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 4.6s
🟢 tests-agent/scenarios.e2e.ts · 21 passed 21 steps · 12 calls 1m 37s
🟢 act completes shadow-dom-form 1 step · 3 calls 4.4s
🟢 act completes lying-labels 1 step · 3 calls 5.7s
🟢 act completes iframe-form 1 step · 3 calls 4.9s
🟢 act completes infinite-scroll 1 step 3.8s
🟢 act completes virtualized-table 1 step 45.3s
🟢 act completes overlay-trap 1 step 827ms
🟢 act completes sticky-chrome 1 step 839ms
🟢 act completes async-states 1 step · 3 calls 7.3s
🟢 act completes debounced-search 1 step 2.3s
🟢 act completes hover-menu 1 step 499ms
🟢 act completes drag-and-drop 1 step 419ms
🟢 act completes kanban-board 1 step 597ms
🟢 act completes context-menu-trap 1 step 2.6s
🟢 act completes stale-dom 1 step 415ms
🟢 act completes otp-auto-advance 1 step 680ms
🟢 act completes date-picker 1 step 749ms
🟢 act completes rich-text-editor 1 step 9.9s
🟢 act completes playbook-cleanup 1 step 530ms
🟢 act completes promo-storefront 1 step 2.3s
🟢 act completes gift-card-purchase 1 step 2.4s
🟢 act filters the list and the URL carries the filter 1 step 227ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 285ms
🟢 act adds an address with a run-unique label 1 step 285ms

Tested by e2e 0.22.0 · 1m 58s · web · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 35 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/migrate/playwright.mdx Resolved
Comment thread packages/e2e/src/report/running-tree.ts Resolved
Comment thread packages/e2e/src/run/step-hold.ts Resolved
Comment thread packages/e2e/src/cli/exit-codes.ts Resolved
@KrzysztofMoch
KrzysztofMoch force-pushed the krzysztof/eng-1077-e2e-run-step branch from a93e515 to d904b4b Compare November 4, 2026 14:33
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 12 replayed from cache · 66 model calls · 438.9k tokens (81% cached) · $0.0307

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 66 calls 9m 50s
🟢 act completes Login Form 1 step 42.0s
🟢 act completes Infinite Scroll List 1 step 31.9s
🟢 act completes Modal Flow 1 step 10.6s
🟢 act completes Bottom Tabs 1 step 7.5s
🟢 act completes Text Input Variations 1 step 13.3s
🟢 act completes Broken Accessibility 2 steps · 16 calls 1m 34s
🟢 act completes Vision Only 2 steps · 6 calls 28.7s
🟢 act completes Gestures 1 step · 7 calls 31.3s
🟢 act completes Async States 1 step · 9 calls 40.0s
🟢 act completes Debounced Search 1 step · 1 call 27.7s
🟢 act completes Flattened Registration Form 2 steps · 1 call 29.9s
🟢 act completes Flattened Login 2 steps · 10 calls 56.9s
🟢 act completes Sticky Chrome Target 1 step 11.0s
🟢 act completes WebView Accessibility 1 step 13.7s
🟢 act completes Permission Prompt 1 step 13.4s
🟢 act completes Photo Picker 1 step · 2 calls 31.9s
🟢 act completes Product Catalog 1 step 10.9s
🟢 act completes Error Recovery 1 step 10.2s
🟢 act completes Choice Controls 1 step · 6 calls 37.3s
🟢 act completes Stripe PaymentSheet 1 step · 8 calls 33.6s
🟢 act completes Apple Pay 1 step 14.5s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.22.0 · 9m 51s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 1 flaky, 19 passed

24 agent steps · 15 replayed from cache · 50 model calls

⚠️ 1 flaky test passed on a retry

⚠️ act completes Modal Flow
tests/fixtures.ts:76

ENGINE_FAILURE at step 1 of 1: app.open, after 2.4s

open dev.e2e.benchmark failed: Android app dev.e2e.benchmark process remained running after close

  • Screen: dev.e2e.benchmark

Evidence: screenshot, log

All 20 tests in 1 file
Test Agent Time
⚠️ tests-agent/scenarios.e2e.ts · 1 flaky, 19 passed 24 steps · 50 calls 5m 38s
🟢 act completes Login Form 1 step 22.8s
🟢 act completes Infinite Scroll List 1 step 22.6s
⚠️ act completes Modal Flow (1 failed attempt first) 1 step 12.2s
🟢 act completes Bottom Tabs 1 step 7.0s
🟢 act completes Text Input Variations 1 step 6.5s
🟢 act completes Broken Accessibility 2 steps · 5 calls 15.4s
🟢 act completes Vision Only 2 steps · 1 call 6.7s
🟢 act completes Gestures 1 step · 10 calls 27.8s
🟢 act completes Async States 1 step · 22 calls 50.6s
🟢 act completes Debounced Search 1 step 6.9s
🟢 act completes Huge Virtualized List 1 step 1m 3s
🟢 act completes Flattened Registration Form 2 steps · 1 call 9.1s
🟢 act completes Flattened Login 2 steps · 6 calls 21.9s
🟢 act completes Sticky Chrome Target 1 step 7.4s
🟢 act completes WebView Accessibility 1 step · 5 calls 15.9s
🟢 act completes Permission Prompt 1 step 6.4s
🟢 act completes Product Catalog 1 step 8.3s
🟢 act completes Error Recovery 1 step 8.8s
🟢 act completes Choice Controls 1 step 8.5s
🟢 act completes Sequential Onboarding 1 step 9.6s

Tested by e2e 0.22.0 · 5m 45s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 2m 59s
🟢 control inventory › a label query resolves a labeled text field 11.5s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 17.3s
🟢 control inventory › a placeholder query and attribute reads find the field 5.4s
🟢 control inventory › a secure field takes a fill and denies value reads 9.0s
🟢 control inventory › press sends a character and Enter through the soft keyboard 8.5s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 10.1s
🟢 control inventory › toBeFocused follows the focus 9.3s
🟢 control inventory › a switch checks, unchecks, and reports its state 10.2s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 8.0s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 12.6s
🟢 control inventory › a radio group reports the checked option 8.8s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 7.8s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.2s
🟢 control inventory › a header answers its accessible name 4.3s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.2s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.6s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.2s
🟢 control inventory › pointer verbs land at the point asked for 9.8s
🟢 control inventory › an ambiguous locator fails at once 5.7s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 11.0s
🟢 refusals › verbs a device never declares are refused before the engine 1ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 445ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 33s
🟢 device fixture › setAppearance flips the color scheme the app reads 12.3s
🟢 device fixture › setOrientation rotates the window the app measures 9.3s
🟢 device fixture › the clipboard reads back what was written 6.9s
🟢 device fixture › home leaves the app, openApp brings it back where it was 7.5s
🟢 device fixture › closeApp then openApp relaunches on the home list 8.0s
🟢 device fixture › closeApp leaves the next commands on the same device 10.9s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 7.1s
🟢 device fixture › device.back pops the scenario 6.4s
🟢 device fixture › unstable_raw reaches the test session through agent-device 4.8s
🟢 device fixture › setLocation and clearLocation change what the app reads 10.5s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 9.8s
🟢 tests/device.e2e.ts · 3 passed 1m 6s
🟢 sequential onboarding echoes the exact values 40.5s
🟢 onboarding rejects a malformed email before advancing 13.6s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.5s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 6s
🟢 infinite scroll reaches the target item 22.7s
🟢 product catalog adds exactly two of the right variant 10.0s
🟢 gestures: long-press, then swipe left 9.2s
🟢 sticky chrome: accept below the fold, then continue 10.1s
🟢 async states: load, pull to refresh, claim through the toast 12.0s
🟢 huge virtualized list reaches row 512 1m 50s
🟢 web view coupon form applies the code shown on the page 11.8s
🟢 tests/flows.e2e.ts · 7 passed 1m 9s
🟢 text inputs unlock the submit button 13.3s
🟢 filled text inputs report their values 10.8s
🟢 modal flow confirms through the native alert 9.6s
🟢 bottom tabs act inside the last tab 7.5s
🟢 error recovery retries, then confirms the delete 9.3s
🟢 choice controls place the exact order 7.3s
🟢 debounced search selects the target once results arrive 11.2s
🟢 tests/login-form.e2e.ts · 3 passed 47.1s
🟢 login form › rejects a malformed email 13.6s
🟢 login form › rejects an unknown account 13.7s
🟢 login form › signs in and logs out 19.8s
🟢 tests/smoke.e2e.ts · 1 passed 7.3s
🟢 home lists the scenarios and opens one 7.3s

Tested by e2e 0.22.0 · 5m 46s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android: 57 passed, 1 skipped

All 58 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 47s
🟢 control inventory › a label query resolves a labeled text field 29.8s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 11.7s
🟢 control inventory › a placeholder query and attribute reads find the field 5.7s
🟢 control inventory › a secure field takes a fill and denies value reads 5.5s
🟢 control inventory › press sends a character and Enter through the soft keyboard 7.4s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 7.9s
🟢 control inventory › toBeFocused follows the focus 8.3s
🟢 control inventory › a switch checks, unchecks, and reports its state 8.2s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 6.8s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 10.2s
🟢 control inventory › a tab role answers getByRole 3.7s
🟢 control inventory › a radio group reports the checked option 7.5s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.8s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.6s
🟢 control inventory › a header answers its accessible name 4.0s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.4s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.2s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 6.7s
🟢 control inventory › a double tap lands inside the 300 ms window 6.4s
🟢 control inventory › an ambiguous locator fails at once 4.4s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 6.8s
🟢 refusals › verbs a device never declares are refused before the engine 3ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.3s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 15s
🟢 device fixture › setOrientation rotates the window the app measures 5.0s
🟢 device fixture › home leaves the app, openApp brings it back where it was 4.3s
🟢 device fixture › foregroundApp sees the home screen after home() 3.9s
🟢 device fixture › closeApp then openApp relaunches on the home list 6.6s
🟢 device fixture › closeApp leaves the next commands on the same device 7.0s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 4.2s
🟢 device fixture › device.back pops the scenario 4.6s
🟢 device fixture › unstable_raw reaches the test session through agent-device 3.6s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 15.3s
🟢 device fixture › setLocation and clearLocation change what the app reads 20.1s
🟢 tests/device.e2e.ts · 5 passed 37.0s
🟢 permission prompt › allowing the system dialog enables the microphone 5.5s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 11.2s
🟢 sequential onboarding echoes the exact values 9.2s
🟢 onboarding rejects a malformed email before advancing 5.2s
🟢 dismissing the keyboard over choice controls leaves them as they were 6.0s
🟢 tests/flows-lists.e2e.ts · 7 passed 1m 54s
🟢 infinite scroll reaches the target item 16.2s
🟢 product catalog adds exactly two of the right variant 8.3s
🟢 gestures: long-press, then swipe left 7.6s
🟢 gestures: double-tap completes the flow 9.4s
🟢 sticky chrome: accept below the fold, then continue 9.0s
🟢 async states: load, pull to refresh, claim through the toast 11.3s
🟢 huge virtualized list reaches row 512 52.1s
🟢 tests/flows.e2e.ts · 7 passed 47.3s
🟢 text inputs unlock the submit button 5.5s
🟢 filled text inputs report their values 5.1s
🟢 modal flow confirms through the native alert 8.3s
🟢 bottom tabs act inside the last tab 6.2s
🟢 error recovery retries, then confirms the delete 8.2s
🟢 choice controls place the exact order 5.7s
🟢 debounced search selects the target once results arrive 8.3s
🟢 tests/login-form.e2e.ts · 3 passed 26.0s
🟢 login form › rejects a malformed email 6.9s
🟢 login form › rejects an unknown account 7.1s
🟢 login form › signs in and logs out 12.0s
🟢 tests/smoke.e2e.ts · 1 passed 5.3s
🟢 home lists the scenarios and opens one 5.3s

Tested by e2e 0.22.0 · 7m 52s · android-emulator · run artifacts

A person can now step through a test: --step runs the first step, then holds each later top-level step until Enter, with the --watch keys (n to the next test, a for the rest, h for help, q to quit). The test clock stops while a step is held, so held time counts toward no timeout.
@okwasniewski

Copy link
Copy Markdown
Member

lgtm!

@okwasniewski
okwasniewski merged commit 93c93f5 into main Nov 9, 2026
30 checks passed
@okwasniewski
okwasniewski deleted the krzysztof/eng-1077-e2e-run-step branch November 9, 2026 17:06
@github-actions github-actions Bot mentioned this pull request Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:XXL 1,000+ changed lines, not counting tests, fixtures, translated docs, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e run --step: hold each step until a key is pressed

2 participants