Skip to content

Expand Maestro smoke flow fidelity#16202

Draft
JorgeMucientes wants to merge 13 commits into
feature/maestro-smoke-buildkitefrom
feature/maestro-smoke-fidelity
Draft

Expand Maestro smoke flow fidelity#16202
JorgeMucientes wants to merge 13 commits into
feature/maestro-smoke-buildkitefrom
feature/maestro-smoke-fidelity

Conversation

@JorgeMucientes

@JorgeMucientes JorgeMucientes commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Description

Expands the Maestro smoke stack with higher-fidelity P2 coverage on top of the Buildkite wiring PR while keeping local setup UI-credential-only.

This PR now removes the default REST seed/cleanup path from the runner. The suite exports run-scoped fixture names, creates test data through the app UI, and intentionally leaves those products/orders/coupons on the selected test store.

It also adds or updates:

  • device media fixture setup for product media coverage
  • reusable order/product subflows for suite-created and test-store data
  • expanded order creation, order detail, product detail, product media, hub menu, coupons, and POS coupon/search coverage
  • a quarantined IPP toggle flow for the payments hub

Test Steps

  1. Populate .maestro/.env.local with the selected store URL, email, and password plus any login-flow-specific variables from .maestro/env.example.
  2. Run the offline smoke-suite validation:
ruby -ryaml -e 'ARGV.each { |f| YAML.load_stream(File.read(f)) }; puts "yaml ok"' .maestro/flows/*.yaml .maestro/subflows/*.yaml .maestro/smoke-coverage.yaml
bash -n .maestro/scripts/run-smoke-tests.sh .buildkite/commands/run-maestro-tests.sh
python3 .maestro/scripts/check-smoke-coverage.py
python3 -m py_compile .maestro/scripts/seed-fixtures.py .maestro/scripts/check-smoke-coverage.py
  1. For a device pass, run the Maestro wrapper against a test store. No WooCommerce REST consumer key or cleanup phase is required.

Images/gif

N/A

  • I have considered if this change warrants release notes and have added them to RELEASE-NOTES.txt if necessary. Use the "[Internal]" label for non-user-facing changes.

@JorgeMucientes JorgeMucientes added this to the 25.2 milestone Jul 5, 2026
@JorgeMucientes JorgeMucientes added type: task An internally driven task. category: tooling Anything that involves building & maintaining the project, including scripts, `Fastfile`, etc. category: ui tests Related to UI testing. labels Jul 5, 2026
@dangermattic

Copy link
Copy Markdown
Collaborator
1 Warning
⚠️ This PR is larger than 300 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.
1 Message
📖 This PR is still a Draft: some checks will be skipped.

Generated by 🚫 Danger

@wpmobilebot

wpmobilebot commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

App Icon📲 You can test the changes from this Pull Request in WooCommerce Android by scanning the QR code below to install the corresponding build.

App NameWooCommerce Android
Platform📱 Mobile
FlavorJalapeno
Build TypeDebug
Build Number766
Version25.1-rc-1
Application IDcom.woocommerce.android.prealpha
Commitecd7ad9
Installation URL11fsf6dk2dt68
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@JorgeMucientes
JorgeMucientes force-pushed the feature/maestro-smoke-buildkite branch 6 times, most recently from 44581ee to 7d0bd02 Compare July 8, 2026 15:18
@wpmobilebot wpmobilebot modified the milestones: 25.2, 25.3 Jul 10, 2026
@wpmobilebot

Copy link
Copy Markdown
Collaborator

Version 25.2 has now entered code-freeze, so the milestone of this PR has been updated to 25.3.

@JorgeMucientes
JorgeMucientes force-pushed the feature/maestro-smoke-buildkite branch 12 times, most recently from 167756c to e9d7898 Compare July 15, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: tooling Anything that involves building & maintaining the project, including scripts, `Fastfile`, etc. category: ui tests Related to UI testing. type: task An internally driven task.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants