Skip to content

feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init - #1647

Merged
okwasniewski merged 16 commits into
mainfrom
kacper/eng-1084-add-cicd-for-e2e-init
Nov 16, 2026
Merged

okwasniewski merged 16 commits into
mainfrom
kacper/eng-1084-add-cicd-for-e2e-init

Conversation

@kacperkapusciak

@kacperkapusciak kacperkapusciak commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

e2e init gets a CI step. In a project with the web or mobile engine it asks which CI service to set up, then writes one job for each platform the config's targets run on: web, iOS, and Android.

◆  Add a CI pipeline that runs the tests?
│  ○ GitHub Actions
│  ● GitLab CI/CD (in every merge request and on the default branch)
│  ○ None
Targets in the config Jobs
web only web, running every target
iOS and/or Android only ios and/or android
web and mobile all of them, each with --target <its targets>
  • --ci github-actions|gitlab|none answers the question. --yes alone writes none, since a pipeline runs on the next push. The wizard preselects the service the repository points to: its remote, a .github/ folder, or a .gitlab-ci.yml.
  • Targets come from the new config init writes, or from loading an existing one the way e2e run does (loadConfigModule, and resolvePlatform/targetName from config/targets.ts). The load happens after the user confirms and after the install, so the config never runs on a cancelled wizard and an example's engine is installed first. A config that still doesn't load falls back to package.json: web engine only, one web job runs every target; mobile engine, no pipeline.
  • Web job: the project's package manager (npm, pnpm, Yarn 1, Yarn 2+, Bun), Chromium with a cache, e2e run --reporter list,junit. It runs every target that isn't iOS or Android, HTTP tests included.
  • Mobile jobs on GitHub follow e2e-inc/e2e-react-native's workflow: macos-26 with a Release simulator build, the image's own iPhone 17 Pro on a pinned iOS runtime, and E2E_DEVICE set to its UDID; ubuntu-latest with KVM, an x86_64 release APK, and an API 36 emulator, with a system-image install that retries a download that drops or comes back empty.
  • The app build: Expo (expo run:ios, expo prebuild + Gradle, no prebuild when android/ is committed), bare React Native (xcodebuild on ios/*.xcworkspace, through Bundler with a Gemfile; android/gradlew), or a step that fails until replaced, which init names.
  • GitLab's hosted runners can't run a simulator or an emulator, so its mobile jobs target self-managed shell runners with tags to replace, as on docs/ci/gitlab.mdx; init says so, and asks for device: process.env.E2E_DEVICE when the config lacks it.
  • Node comes from .nvmrc or .node-version, else 24. A project in a subfolder or a workspace member gets the file at the repository root, the install at the workspace root, and the jobs in the project folder.
  • Model key: GitHub passes the repository secret in the run step's env; GitLab gets CI/CD variables on its own. init reads the gateway from the config's import and prints the gh secret set or glab variable set command.
  • init never overwrites a pipeline file. Beside an existing .gitlab-ci.yml it writes .gitlab/e2e.gitlab-ci.yml and prints the include: to add, renaming the jobs with -run when the pipeline has one of their names and saying when its stages: leave out test. A workflow or pipeline that already runs e2e run in one of its commands gets nothing; the words in a comment or a step name don't count. Nothing is written through a symlinked .github or .gitlab.
  • Warnings for what would fail on the first push: no app.command (web job only), a subscription gateway or a loopback model endpoint, a build step to replace, GitLab mobile runners.
  • The pipelines are YAML templates in packages/e2e/templates/ci/, each a complete pipeline for npm at the repository root and an Expo app, which actionlint and glab ci lint accept as is. init edits a copy with the yaml package's Document API (new runtime dependency, pure JS), keeping comments and formatting; a platform with no targets loses its job. For the template's own case the output equals the template byte for byte.
  • GitHub action pins live only in the templates and match the docs and the e2e skill; tests/unit/init-ci.test.ts fails when they drift.
  • Telemetry: e2e_init_completed gets a ci property.
flowchart TD
  A[web or mobile engine in package.json?] -- no --> N[no question]
  A -- yes --> B{e2e run already in a workflow or .gitlab-ci.yml?}
  B -- yes --> N
  B -- no --> C[ask, preselect from remote / .github / .gitlab-ci.yml]
  C --> D[confirm changes, install]
  D --> E[targets: new config, or load the existing one]
  E --> F[web job for non-phone targets, ios and android jobs for phone targets]
  F --> G[.github/workflows/e2e.yml, .gitlab-ci.yml, or .gitlab/e2e.gitlab-ci.yml to include]
Loading

Verified

Ran it locally: yes. The built CLI wrote the pipelines into real repos, and each ran unedited on hosted runners:

Case Where Result
GitHub, web + iOS + Android, Expo, existing config with three targets e2e-react-native#5, its own workflow replaced by the generated one web passed; iOS passed (35/35, 51 min) on the first run; Android passed (23 min) on the final template. On that last run iOS lost 1 of 35 to a simulator xcrun timeout, with its job unchanged since the green run
GitHub Actions, npm, repo root e2e-github-actions#2, merged 5/5, agent tests included
GitHub Actions, pnpm in site/, .nvmrc at the root e2e-github-actions#3 5/5
GitHub Actions, pnpm version from site/package.json e2e-github-actions#5 passed
GitHub, web job from the YAML template e2e-github-actions#6 5/5, cache replayed
GitLab, npm, new .gitlab-ci.yml e2e-gitlab-ci!1, merged 5/5 in the MR test report
GitLab, beside an existing .gitlab-ci.yml via include: e2e-gitlab-ci!2 5/5
GitLab, pnpm via corepack on node:26 e2e-gitlab-ci!5 5/5
GitLab, web job from the YAML template e2e-gitlab-ci!6 5/5
  • actionlint and glab ci lint pass on every combination rendered: web only, mobile only, both, each build kind, npm/pnpm/Yarn/Bun, a subfolder, and a workspace member. The GitLab file for e2e-react-native passes glab ci lint too.
  • Packed tarball in a scratch project, and the interactive wizard in a pseudo-terminal: the question above, with the service preselected from the remote.
  • The Android job on GitHub needed two fixes found on that repo: sdkmanager can report success for an empty image, and the runner image's command-line tools (since image 20261103) are older than the API 36 system image's package format, so android-emulator-runner failed to create the device ("contains no system images"). The job now installs command-line tools 23.0 (SHA-256 checked) and checks the image before the action runs.
  • Built CLI and packed tarball in scratch repos: beside a Cypress workflow whose step is named Cypress e2e run and commented # our old e2e runner, --ci github-actions wrote .github/workflows/e2e.yml (before: Exists, not touching: .github/workflows/cypress.yml); beside a workflow whose multi-line step runs xvfb-run -a npx e2e run, it wrote nothing.
  • Not run on a runner: the bare React Native build, and GitLab's mobile jobs (gitlab.com's hosted runners can't run them).

Two fresh-context reviews found issues, all fixed: the first nine in the web-only version, then eight in the mobile one (the config loaded before the install, so an example got no pipeline; the GitLab header lost with the web job; and smaller ones). cubic's findings are fixed or answered in their threads, and kdy1's suggestion to keep the pipelines as YAML templates is in.

Tracked in ENG-1084.

@github-actions github-actions Bot added the size:XL Up to 999 changed lines, not counting tests, fixtures, translated docs, or the lockfile label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 93 passed

All 93 tests in 42 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 896ms
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 667ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 229ms
🟢 tests/async-states.e2e.ts · 2 passed 7.1s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.3s
🟢 async states › a wrong code is rejected after the toast has gone 4.8s
🟢 tests/canvas-only.e2e.ts · 2 passed 424ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 219ms
🟢 canvas only › a shape out of order paints the error and resets progress 205ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 586ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 315ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 271ms
🟢 tests/cart-totals.e2e.ts · 2 passed 494ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 267ms
🟢 cart totals › a quantity never drops below one 227ms
🟢 tests/checkout-review.e2e.ts · 1 passed 233ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 233ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 718ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 375ms
🟢 context menu trap › the decoy menu items refuse and keep the file 343ms
🟢 tests/control-inventory.e2e.ts · 8 passed 4.1s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 333ms
🟢 control inventory › check picks a radio the pick replaces with its summary 248ms
🟢 control inventory › a long press and a double click are told apart from a tap 1.0s
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 303ms
🟢 control inventory › setInputFiles attaches files named from the project root 190ms
🟢 control inventory › the details view is left through the browser history 229ms
🟢 control inventory › scrollIntoView brings the footnote into view 310ms
🟢 control inventory › every exercise done shows the success message 1.4s
🟢 tests/css-content-ui.e2e.ts · 2 passed 426ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 241ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 185ms
🟢 tests/date-picker.e2e.ts · 2 passed 759ms
🟢 date picker › navigates three months ahead and books the required date 445ms
🟢 date picker › confirming without the exact date shows the error 314ms
🟢 tests/debounced-search.e2e.ts · 2 passed 2.9s
🟢 debounced search › a decoy one character off is refused 1.5s
🟢 debounced search › picking the exact item adds it to the cart 1.5s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.1s
🟢 one submit opens one confirm, even when double-clicked 1.1s
🟢 tests/div-soup.e2e.ts · 2 passed 873ms
🟢 div soup › a wrong code is rejected 388ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 485ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 715ms
🟢 drag and drop › an out-of-order item resets the dropzone 370ms
🟢 drag and drop › dropping Banana then Cherry passes 345ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 299ms
🟢 filter deep link › filtering writes the query to the URL 175ms
🟢 filter deep link › opening a shared link restores the filter 124ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 748ms
🟢 gift card purchase › refuses to buy without an amount 209ms
🟢 gift card purchase › refuses a recipient without an at sign 285ms
🟢 gift card purchase › sends the gift card 254ms
🟢 tests/hover-menu.e2e.ts · 2 passed 576ms
🟢 hover menu › a decoy leaf is refused 263ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 313ms
🟢 tests/iframe-form.e2e.ts · 2 passed 576ms
🟢 iframe form › a wrong code is refused by the host page 261ms
🟢 iframe form › the code read from one frame applies through the other 315ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 556ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 121ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 187ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 248ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 2.9s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 2.9s
🟢 tests/kanban-board.e2e.ts · 3 passed 796ms
🟢 kanban board › pointer drags move both cards into the goal layout 398ms
🟢 kanban board › a swipe along a path drags a card as well 183ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 215ms
🟢 tests/login-form.e2e.ts · 4 passed 863ms
🟢 login form › rejects a malformed email 205ms
🟢 login form › rejects a short password 192ms
🟢 login form › rejects an unknown account 207ms
🟢 login form › signs in and logs out 259ms
🟢 tests/lying-labels.e2e.ts · 4 passed 785ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 185ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 114ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 248ms
🟢 lying labels › visible text drives the payment through 238ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 582ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 197ms
🟢 native dialogs › dismissing the confirm cancels the order 186ms
🟢 native dialogs › a wrong gift code is rejected 199ms
🟢 tests/new-tab-flow.e2e.ts · 2 passed 461ms
🟢 new tab flow › reads the code in the tab the app opens and enters it back in the original tab 239ms
🟢 new tab flow › a tap that opens a tab leaves the screen on the original tab, and activate shows the new one 222ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.3s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.3s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 5.9s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 368ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 418ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.6s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 1.7s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 1.8s
🟢 tests/order-history.e2e.ts · 2 passed 1.5s
🟢 order history › overview and settings render their panels 237ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.3s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.6s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 532ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 192ms
🟢 otp auto-advance › a wrong code clears every box and starts over 857ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 743ms
🟢 continues only after the banner and the overlay are gone 743ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 731ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 396ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 335ms
🟢 tests/price-sorting.e2e.ts · 3 passed 483ms
🟢 price sorting › opens in the featured order with sale prices struck through 147ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 157ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 179ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.2s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.2s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.4s
🟢 buys a deal once the promo chrome is out of the way 1.4s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 729ms
🟢 rich text editor › bolding exactly the required word passes the check 319ms
🟢 rich text editor › bold that spills into the previous word fails the check 410ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 802ms
🟢 saved addresses › adds an address and clears the form 279ms
🟢 saved addresses › ignores an add with an empty field 253ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 270ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 590ms
🟢 shadow DOM form › rejects a wrong access code 361ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 229ms
🟢 tests/smoke.e2e.ts · 1 passed 417ms
🟢 home lists the scenarios and opens one 417ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.2s
🟢 taps the moving target three times across rebuilds of the list 2.2s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 304ms
🟢 accepts the terms once the button clears the sticky footer 304ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 18.1s
🟢 scrolls the windowed table to the Golden Row and claims it 18.1s
🟢 tests/wishlist.e2e.ts · 2 passed 1.8s
🟢 wishlist › saving toggles the button and the tab badge 358ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.5s

Tested by e2e 0.23.1 · 1m 12s · web · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 34 passed

35 agent steps · 26 replayed from cache · 25 model calls · 117.2k tokens (94% cached) · $0.0046

All 34 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.9s
🟢 control inventory › check sets the checkbox and the radio 1 step 858ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 290ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 973ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.7s
🟢 control inventory › upload attaches the named project files 1 step 2.5s
🟢 control inventory › back returns from the details view to the inventory 1 step 322ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 373ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 449ms
🟢 act signs in with the benchmark account 1 step 449ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 12.0s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 6.4s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.6s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 4.1s
🟢 tests-agent/scenarios.e2e.ts · 22 passed 22 steps · 17 calls 1m 22s
🟢 act completes shadow-dom-form 1 step · 3 calls 3.6s
🟢 act completes lying-labels 1 step · 3 calls 4.4s
🟢 act completes iframe-form 1 step · 3 calls 4.1s
🟢 act completes infinite-scroll 1 step 3.5s
🟢 act completes virtualized-table 1 step 22.6s
🟢 act completes overlay-trap 1 step 973ms
🟢 act completes sticky-chrome 1 step 961ms
🟢 act completes async-states 1 step · 3 calls 6.0s
🟢 act completes debounced-search 1 step 2.4s
🟢 act completes hover-menu 1 step 632ms
🟢 act completes drag-and-drop 1 step 585ms
🟢 act completes kanban-board 1 step 773ms
🟢 act completes context-menu-trap 1 step 3.0s
🟢 act completes stale-dom 1 step 841ms
🟢 act completes otp-auto-advance 1 step 1.1s
🟢 act completes date-picker 1 step 1.3s
🟢 act completes rich-text-editor 1 step 11.1s
🟢 act completes playbook-cleanup 1 step 852ms
🟢 act completes promo-storefront 1 step 1.5s
🟢 act completes new-tab-flow 1 step · 5 calls 8.7s
🟢 act completes gift-card-purchase 1 step 2.6s
🟢 act filters the list and the URL carries the filter 1 step 479ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 415ms
🟢 act adds an address with a run-unique label 1 step 415ms

Tested by e2e 0.23.1 · 1m 44s · web · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Comment thread skills/e2e/references/setup.md Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/telemetry/events.ts Resolved
Comment thread docs/reference/cli.mdx Resolved
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 1 flaky, 20 passed, 1 skipped

25 agent steps · 11 replayed from cache · 67 model calls · 474.1k tokens (81% cached) · $0.0336

⚠️ 1 flaky test passed on a retry

⚠️ act completes Gestures
tests-agent/scenarios.e2e.ts:275

ASSERTION_FAILED at step 5 of 5: expect.toHaveText getByTestId("success-message"), after 5.0s

  • Expected: text "All gestures completed"
  • Observed: no node (0 matches)
  • Screen: dev.e2e.benchmark / Gestures

Evidence: screenshot, log

All 22 tests in 1 file
Test Agent Time
⚠️ tests-agent/scenarios.e2e.ts · 1 flaky, 20 passed, 1 skipped 25 steps · 67 calls 9m 19s
🟢 act completes Login Form 1 step · 8 calls 1m 14s
🟢 act completes Infinite Scroll List 1 step 38.0s
🟢 act completes Modal Flow 1 step 10.7s
🟢 act completes Bottom Tabs 1 step 7.6s
🟢 act completes Text Input Variations 1 step 12.9s
🟢 act completes Broken Accessibility 2 steps · 10 calls 1m 6s
🟢 act completes Vision Only 2 steps · 6 calls 28.4s
⚠️ act completes Gestures (1 failed attempt first) 1 step · 9 calls 35.2s
🟢 act completes Async States 1 step · 11 calls 43.5s
🟢 act completes Debounced Search 1 step · 1 call 26.6s
🟢 act completes Flattened Registration Form 2 steps · 1 call 25.9s
🟢 act completes Flattened Login 2 steps · 10 calls 51.9s
🟢 act completes Sticky Chrome Target 1 step 12.9s
🟢 act completes WebView Accessibility 1 step 12.5s
🟢 act completes Permission Prompt 1 step 11.4s
🟢 act completes Photo Picker 1 step · 2 calls 30.3s
🟢 act completes Product Catalog 1 step 10.2s
🟢 act completes Error Recovery 1 step 9.5s
🟢 act completes Choice Controls 1 step · 3 calls 27.6s
🟢 act completes Stripe PaymentSheet 1 step · 6 calls 12.6s
🟢 act completes Apple Pay 1 step 11.0s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.1 · 10m 18s · ios-simulator · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 9 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 20 passed

24 agent steps · 15 replayed from cache · 41 model calls · 231.2k tokens (84% cached) · $0.0145

All 20 tests in 1 file
Test Agent Time
🟢 tests-agent/scenarios.e2e.ts · 20 passed 24 steps · 41 calls 6m 22s
🟢 act completes Login Form 1 step 30.8s
🟢 act completes Infinite Scroll List 1 step 21.2s
🟢 act completes Modal Flow 1 step 10.5s
🟢 act completes Bottom Tabs 1 step 7.1s
🟢 act completes Text Input Variations 1 step 6.4s
🟢 act completes Broken Accessibility 2 steps · 6 calls 21.3s
🟢 act completes Vision Only 2 steps · 1 call 7.5s
🟢 act completes Gestures 1 step · 9 calls 24.4s
🟢 act completes Async States 1 step · 12 calls 30.9s
🟢 act completes Debounced Search 1 step 9.6s
🟢 act completes Huge Virtualized List 1 step 1m 54s
🟢 act completes Flattened Registration Form 2 steps · 1 call 9.1s
🟢 act completes Flattened Login 2 steps · 6 calls 15.4s
🟢 act completes Sticky Chrome Target 1 step 10.1s
🟢 act completes WebView Accessibility 1 step · 6 calls 19.3s
🟢 act completes Permission Prompt 1 step 6.9s
🟢 act completes Product Catalog 1 step 8.7s
🟢 act completes Error Recovery 1 step 9.8s
🟢 act completes Choice Controls 1 step 9.2s
🟢 act completes Sequential Onboarding 1 step 10.4s

Tested by e2e 0.23.1 · 6m 24s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 56 passed, 1 skipped

All 57 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 3m 10s
🟢 control inventory › a label query resolves a labeled text field 10.7s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 17.0s
🟢 control inventory › a placeholder query and attribute reads find the field 9.4s
🟢 control inventory › a secure field takes a fill and denies value reads 8.9s
🟢 control inventory › press sends a character and Enter through the soft keyboard 9.6s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 11.6s
🟢 control inventory › toBeFocused follows the focus 9.9s
🟢 control inventory › a switch checks, unchecks, and reports its state 11.0s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 8.7s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 13.7s
🟢 control inventory › a radio group reports the checked option 8.9s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 7.9s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.5s
🟢 control inventory › a header answers its accessible name 4.4s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.4s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.8s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.4s
🟢 control inventory › pointer verbs land at the point asked for 10.2s
🟢 control inventory › an ambiguous locator fails at once 5.7s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 11.4s
🟢 refusals › verbs a device never declares are refused before the engine 3ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 440ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 12 passed 1m 57s
🟢 device fixture › setAppearance flips the color scheme the app reads 11.1s
🟢 device fixture › setOrientation rotates the window the app measures 6.9s
🟢 device fixture › the clipboard reads back what was written 6.5s
🟢 device fixture › home leaves the app, openApp brings it back where it was 8.4s
🟢 device fixture › closeApp then openApp relaunches on the home list 9.3s
🟢 device fixture › closeApp leaves the next commands on the same device 14.6s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 9.1s
🟢 device fixture › openLink opens the scenario a deep link names 18.1s
🟢 device fixture › device.back pops the scenario 6.9s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 4.2s
🟢 device fixture › setLocation and clearLocation change what the app reads 11.2s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 11.1s
🟢 tests/device.e2e.ts · 3 passed 48.7s
🟢 sequential onboarding echoes the exact values 26.7s
🟢 onboarding rejects a malformed email before advancing 10.4s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.6s
🟢 tests/flows-lists.e2e.ts · 7 passed 4m 16s
🟢 infinite scroll reaches the target item 34.7s
🟢 product catalog adds exactly two of the right variant 10.5s
🟢 gestures: long-press, then swipe left 8.9s
🟢 sticky chrome: accept below the fold, then continue 12.7s
🟢 async states: load, pull to refresh, claim through the toast 13.1s
🟢 huge virtualized list reaches row 512 2m 42s
🟢 web view coupon form applies the code shown on the page 14.0s
🟢 tests/flows.e2e.ts · 7 passed 1m 10s
🟢 text inputs unlock the submit button 12.4s
🟢 filled text inputs report their values 11.0s
🟢 modal flow confirms through the native alert 9.9s
🟢 bottom tabs act inside the last tab 7.6s
🟢 error recovery retries, then confirms the delete 9.7s
🟢 choice controls place the exact order 7.9s
🟢 debounced search selects the target once results arrive 11.5s
🟢 tests/login-form.e2e.ts · 3 passed 45.9s
🟢 login form › rejects a malformed email 12.7s
🟢 login form › rejects an unknown account 13.0s
🟢 login form › signs in and logs out 20.1s
🟢 tests/smoke.e2e.ts · 1 passed 9.3s
🟢 home lists the scenarios and opens one 9.3s

Tested by e2e 0.23.1 · 7m 2s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android: 58 passed, 1 skipped

All 59 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 50s
🟢 control inventory › a label query resolves a labeled text field 24.0s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 11.1s
🟢 control inventory › a placeholder query and attribute reads find the field 6.6s
🟢 control inventory › a secure field takes a fill and denies value reads 8.3s
🟢 control inventory › press sends a character and Enter through the soft keyboard 7.7s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 7.8s
🟢 control inventory › toBeFocused follows the focus 8.5s
🟢 control inventory › a switch checks, unchecks, and reports its state 8.1s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 6.9s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 10.2s
🟢 control inventory › a tab role answers getByRole 4.0s
🟢 control inventory › a radio group reports the checked option 7.4s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.6s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.9s
🟢 control inventory › a header answers its accessible name 4.3s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.7s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.9s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.3s
🟢 control inventory › a double tap lands inside the 300 ms window 6.8s
🟢 control inventory › an ambiguous locator fails at once 5.1s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 7.9s
🟢 refusals › verbs a device never declares are refused before the engine 4ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.2s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 21s
🟢 device fixture › setOrientation rotates the window the app measures 5.1s
🟢 device fixture › home leaves the app, openApp brings it back where it was 4.7s
🟢 device fixture › foregroundApp sees the home screen after home() 4.4s
🟢 device fixture › closeApp then openApp relaunches on the home list 7.2s
🟢 device fixture › closeApp leaves the next commands on the same device 6.3s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 4.0s
🟢 device fixture › openLink opens the scenario a deep link names 4.6s
🟢 device fixture › device.back pops the scenario 4.4s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 3.8s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 15.1s
🟢 device fixture › setLocation and clearLocation change what the app reads 21.2s
🟢 tests/device.e2e.ts · 5 passed 40.2s
🟢 permission prompt › allowing the system dialog enables the microphone 5.8s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 12.0s
🟢 sequential onboarding echoes the exact values 9.7s
🟢 onboarding rejects a malformed email before advancing 5.7s
🟢 dismissing the keyboard over choice controls leaves them as they were 7.0s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 5s
🟢 infinite scroll reaches the target item 27.7s
🟢 product catalog adds exactly two of the right variant 8.3s
🟢 gestures: long-press, then swipe left 7.7s
🟢 gestures: double-tap completes the flow 9.2s
🟢 sticky chrome: accept below the fold, then continue 10.5s
🟢 async states: load, pull to refresh, claim through the toast 11.5s
🟢 huge virtualized list reaches row 512 1m 50s
🟢 tests/flows.e2e.ts · 7 passed 50.5s
🟢 text inputs unlock the submit button 6.5s
🟢 filled text inputs report their values 5.5s
🟢 modal flow confirms through the native alert 9.0s
🟢 bottom tabs act inside the last tab 6.5s
🟢 error recovery retries, then confirms the delete 8.4s
🟢 choice controls place the exact order 6.0s
🟢 debounced search selects the target once results arrive 8.6s
🟢 tests/login-form.e2e.ts · 3 passed 26.7s
🟢 login form › rejects a malformed email 7.2s
🟢 login form › rejects an unknown account 7.2s
🟢 login form › signs in and logs out 12.3s
🟢 tests/smoke.e2e.ts · 1 passed 5.8s
🟢 home lists the scenarios and opens one 5.8s

Tested by e2e 0.23.1 · 9m 19s · android-emulator · run artifacts

Comment thread packages/e2e/src/cli/init/ci.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.
Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.
The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.
@kacperkapusciak
kacperkapusciak force-pushed the kacper/eng-1084-add-cicd-for-e2e-init branch from b3f72d7 to de6bff7 Compare November 9, 2026 13:50

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 4 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • We need to figure out if you use mobile, then we should output GitHub or GitLab Mobile CI/CD. And if you select web, then appropriate. We should also handle the case where you select two of those.

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.
@github-actions github-actions Bot added size:XXL 1,000+ changed lines, not counting tests, fixtures, translated docs, or the lockfile and removed Ready for Human Review size:XL Up to 999 changed lines, not counting tests, fixtures, translated docs, or the lockfile labels Nov 12, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 26 files

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/core/src/config/targets.ts Resolved
Comment thread packages/e2e/templates/ci/gitlab-ci.yml Resolved
Comment thread skills/e2e/references/running.md Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init.ts Resolved
Comment thread packages/e2e/src/cli/init/ci.ts Resolved
Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 6 files (changes from recent commits).

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Turn on auto-fix | Re-trigger cubic

…emulator image

The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it.
init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own.

Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies.
…ates, docs, and skill

docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in.
@okwasniewski
okwasniewski added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit d593863 Nov 16, 2026
29 checks passed
@okwasniewski
okwasniewski deleted the kacper/eng-1084-add-cicd-for-e2e-init branch November 16, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:XXL 1,000+ changed lines, not counting tests, fixtures, translated docs, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants