Skip to content

perf: decode and encode PNGs with @napi-rs/image - #1587

Merged
okwasniewski merged 2 commits into
mainfrom
kdy1/replace-png-libraries
Nov 6, 2026
Merged

okwasniewski merged 2 commits into
mainfrom
kdy1/replace-png-libraries

Conversation

@kdy1

@kdy1 kdy1 commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator
  • e2e, @e2e-dev/mobile, and @e2e-dev/decision read and write PNGs with @napi-rs/image instead of fast-png and pngjs (packages/e2e/src/internal/image.ts, packages/mobile/src/png.ts, packages/decision/src/png.ts). pixelmatch stays: the addon has no diff API.
  • Chromium's captures are 8-bit RGB. The addon widens them to RGBA itself (opacity(1) keeps every sample), which was most of the decode once the codec was native. Other color types go through a short layout step in JS.
  • Decoded pixels match the old libraries on the 76 PNGs the repo tracks and on a synthetic matrix of color types, depths, interlacing, palettes, and transparent colors. Two behaviors change on purpose and are in the changeset: gray PNGs with 1, 2, or 4 bits now decode, and a 16-bit gray or RGB PNG with a transparent color compares at 16 bits.
  • e2e loads the addon when the first PNG is read or written (createRequire, as oxc() does), so --version, --help, and a run that takes no screenshot never need the binding. A missing binding fails the call with the new IMAGE_CODEC_UNAVAILABLE (docs/reference/errors.mdx). The STARTUP table in package-imports.test.ts pins the lazy load.
  • @e2e-dev/mobile and @e2e-dev/decision import the addon with the package. They are opt-in, and a lazy load there would turn a missing binding into a secure screenshot silently withheld (redactSecure) or a zoom silently skipped (decode in overlay.ts). Stopping at import is the louder failure.

Speed

Main (0c1b3572) against this branch: both dist builds in one process, A/B then B/A, 11 rounds per cell, medians over three real screenshots per profile. Each case first checks that both sides produce the same pixels.

web 1280×720 web 2560×1440 (DPR 2) phone 1170×2532 (DPR 3) device 804×1786
decode 30.8 → 2.06 ms 117 → 9.98 ms 97.5 → 7.78 ms 20.4 → 2.64 ms
encode 32.9 → 5.32 ms 134 → 16.4 ms 106 → 19.9 ms 50.8 → 6.66 ms
model-bound downscale, downscalePixels(768) 50.4 → 9.87 ms 147 → 28.0 ms 120 → 23.4 ms 38.8 → 11.2 ms
toHaveScreenshot capture, whole screen 31.1 → 2.51 ms 136 → 29.4 ms
mobile maskPng, 2 rects 197 → 28.5 ms 90.5 → 8.84 ms
decision zoomAround(400, 2x) 39.3 → 5.36 ms 39.4 → 5.38 ms 39.1 → 7.68 ms 37.6 → 5.55 ms

What is left at 2560×1440 is the JS box resample, unchanged here: 14.6 of the 28 ms.

pnpm bench:ab --suite testbed -- tests/visual.e2e.ts, 30 pairs. Every test ran the same steps with the same outcome, and every counter matched. The harness notes that the suite's tests differ between the builds because screenshot.e2e.ts was ported; visual.e2e.ts itself is unchanged.

main this PR change (95% CI)
step expect.toHaveScreenshot 454 ms 340 ms -24.6% (-27.8% to -23.2%)
suite, sum of attempts 2.24 s 2.09 s -6.8% (-7.5% to -5.9%)
wall, process 3.32 s 3.14 s -5.1% (-5.6% to -4.5%)
e2e --version, 30 interleaved pairs 194.7 ms 188.7 ms -3.4%

Decisions for the reviewer

  • Platforms. The addon publishes 12 bindings and oxc, already in e2e, publishes 19. Linux on ppc64, s390x, and riscv64 (gnu and musl), armv7 musl, android arm32, and OpenHarmony have none. There toHaveScreenshot and any agent screenshot the runner has to resize (a side over 768 px) fail with IMAGE_CODEC_UNAVAILABLE, and mobile and decision fail at import. The changeset is a patch; whether this is a **Breaking:** minor is your call.
  • Encoded size. FilterType.Adaptive makes a Chromium capture about 15% larger than fast-png wrote it (55 to 63 KB at 1280×720) and a device screenshot about 20% smaller. "The smaller of no filter and adaptive" would never be larger than fast-png on a Chromium capture, costs about twice the encode time (5.3 to 9.7 ms at 1280×720), and touches the same three files. Not done here.
  • Shared layout code. The step from the addon's pixel layouts to 8-bit RGBA (LAYOUTS, narrow, spread) exists in e2e, mobile, and decision. mobile and decision reach e2e only through its public entries, so sharing it means a new export on e2e or e2e/engine, or a new package. readPngSize is copied in web and mobile for the same reason.
  • Install size. The binary is 15.5 MB on macOS arm64 and 21.1 MB on Linux x64 (8.3 MB download), against 1.25 MB for fast-png, fflate, and iobuffer. pngjs stays through pixelmatch.
  • Memory. The addon frees what a Transformer holds in a finalizer that Node runs a loop turn after V8 collects the owner, and it does not tell V8 about that memory. A loop that calls these functions back to back without yielding keeps about 70 MB per 2560×1440 capture: 120 calls peak at 8.4 GB where main stayed at 253 MB. Real runs yield between calls: worker peaks are 240 MB for visual.e2e.ts (main 315 MB) and 490 MB for screenshot.e2e.ts (main 427 MB).

Verified

Ran it locally: yes

  • The built CLI on the testbed, tests/visual.e2e.ts against the stored screenshots fast-png wrote: 4 passed, and no tracked file changed.
  • The failure path, with the platform binding left out (@napi-rs/image-darwin-arm64 unresolvable, which is what --omit=optional does), on a testbed config without decisionFilter:
    • tests/todos.e2e.ts passes: 7 passed, exit 0.
    • tests/visual.e2e.ts fails every test with exit 3:
 × |web| tests/visual.e2e.ts > screenshots > compares a block with a changing part masked
IMAGE_CODEC_UNAVAILABLE: e2e reads and writes screenshots with @napi-rs/image, and its native binding for darwin-arm64 (@napi-rs/image-darwin-arm64*) is not installed. npm skips it with --omit=optional, or when package-lock.json was written on another platform (npm/cli#4828): install again without --omit=optional, or delete package-lock.json and node_modules and run npm install. The package has no binding for Linux on ppc64, s390x, or riscv64
  • The testbed's own config imports @e2e-dev/decision, so there the same missing binding stops at config load, exit 2: failed to load config .../e2e.config.ts: Cannot find native binding. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). ...
  • The mobile benchmark I could not run here (no device). CI ran it on this head: the iOS simulator passed 55 deterministic tests and 21 agentic ones, the Android emulator 57 and 20, with one retry on Android's Gestures, which flaked the same way on perf: skip the MCP server, prompt library, and Playwright until a command needs them #1576.
  • Not run: the agentic suites live against a model (CI replays their recordings), Windows, and Linux outside CI.

🤖 Generated with Claude Code

e2e, @e2e-dev/mobile, and @e2e-dev/decision read and write PNGs with
@napi-rs/image instead of fast-png and pngjs. Decoding is 8 to 15 times
faster, encoding 5 to 8 times, and a toHaveScreenshot capture 4 to 12 times.

- Chromium's 8-bit RGB captures are widened to RGBA by the addon, not by a
  JS loop. Other color types go through a short layout step.
- e2e loads the addon when the first PNG is read or written, so --version,
  --help, and a run with no screenshot never need its native binding. A
  missing binding fails the call with IMAGE_CODEC_UNAVAILABLE.
- pixelmatch stays: the addon has no diff API.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kdy1
kdy1 requested a review from okwasniewski as a code owner November 6, 2026 05:38
@github-actions github-actions Bot added the size:L Up to 499 changed lines, not counting tests, fixtures, translated docs, or the lockfile label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 91 passed

All 91 tests in 41 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 548ms
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 414ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 134ms
🟢 tests/async-states.e2e.ts · 2 passed 7.0s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.2s
🟢 async states › a wrong code is rejected after the toast has gone 4.8s
🟢 tests/canvas-only.e2e.ts · 2 passed 229ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 124ms
🟢 canvas only › a shape out of order paints the error and resets progress 105ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 346ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 194ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 152ms
🟢 tests/cart-totals.e2e.ts · 2 passed 369ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 227ms
🟢 cart totals › a quantity never drops below one 142ms
🟢 tests/checkout-review.e2e.ts · 1 passed 135ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 135ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 494ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 263ms
🟢 context menu trap › the decoy menu items refuse and keep the file 231ms
🟢 tests/control-inventory.e2e.ts · 8 passed 3.5s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 239ms
🟢 control inventory › check picks a radio the pick replaces with its summary 201ms
🟢 control inventory › a long press and a double click are told apart from a tap 991ms
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 238ms
🟢 control inventory › setInputFiles attaches files named from the project root 135ms
🟢 control inventory › the details view is left through the browser history 159ms
🟢 control inventory › scrollIntoView brings the footnote into view 237ms
🟢 control inventory › every exercise done shows the success message 1.3s
🟢 tests/css-content-ui.e2e.ts · 2 passed 363ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 195ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 168ms
🟢 tests/date-picker.e2e.ts · 2 passed 610ms
🟢 date picker › navigates three months ahead and books the required date 343ms
🟢 date picker › confirming without the exact date shows the error 267ms
🟢 tests/debounced-search.e2e.ts · 2 passed 2.8s
🟢 debounced search › a decoy one character off is refused 1.4s
🟢 debounced search › picking the exact item adds it to the cart 1.4s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.1s
🟢 one submit opens one confirm, even when double-clicked 1.1s
🟢 tests/div-soup.e2e.ts · 2 passed 604ms
🟢 div soup › a wrong code is rejected 271ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 333ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 632ms
🟢 drag and drop › an out-of-order item resets the dropzone 301ms
🟢 drag and drop › dropping Banana then Cherry passes 331ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 229ms
🟢 filter deep link › filtering writes the query to the URL 134ms
🟢 filter deep link › opening a shared link restores the filter 95ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 504ms
🟢 gift card purchase › refuses to buy without an amount 136ms
🟢 gift card purchase › refuses a recipient without an at sign 183ms
🟢 gift card purchase › sends the gift card 185ms
🟢 tests/hover-menu.e2e.ts · 2 passed 457ms
🟢 hover menu › a decoy leaf is refused 228ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 229ms
🟢 tests/iframe-form.e2e.ts · 2 passed 510ms
🟢 iframe form › a wrong code is refused by the host page 245ms
🟢 iframe form › the code read from one frame applies through the other 265ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 537ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 124ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 182ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 231ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 2.7s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 2.7s
🟢 tests/kanban-board.e2e.ts · 3 passed 772ms
🟢 kanban board › pointer drags move both cards into the goal layout 384ms
🟢 kanban board › a swipe along a path drags a card as well 176ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 212ms
🟢 tests/login-form.e2e.ts · 4 passed 858ms
🟢 login form › rejects a malformed email 165ms
🟢 login form › rejects a short password 201ms
🟢 login form › rejects an unknown account 204ms
🟢 login form › signs in and logs out 288ms
🟢 tests/lying-labels.e2e.ts · 4 passed 846ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 194ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 142ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 241ms
🟢 lying labels › visible text drives the payment through 269ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 609ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 210ms
🟢 native dialogs › dismissing the confirm cancels the order 216ms
🟢 native dialogs › a wrong gift code is rejected 183ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.3s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.3s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 4.6s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 307ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 384ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.2s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 1.3s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 1.4s
🟢 tests/order-history.e2e.ts · 2 passed 1.5s
🟢 order history › overview and settings render their panels 228ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.3s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.4s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 463ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 139ms
🟢 otp auto-advance › a wrong code clears every box and starts over 788ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 689ms
🟢 continues only after the banner and the overlay are gone 689ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 409ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 209ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 200ms
🟢 tests/price-sorting.e2e.ts · 3 passed 336ms
🟢 price sorting › opens in the featured order with sale prices struck through 98ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 113ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 125ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.1s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.1s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.4s
🟢 buys a deal once the promo chrome is out of the way 1.4s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 427ms
🟢 rich text editor › bolding exactly the required word passes the check 210ms
🟢 rich text editor › bold that spills into the previous word fails the check 217ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 463ms
🟢 saved addresses › adds an address and clears the form 151ms
🟢 saved addresses › ignores an add with an empty field 147ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 165ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 518ms
🟢 shadow DOM form › rejects a wrong access code 249ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 269ms
🟢 tests/smoke.e2e.ts · 1 passed 392ms
🟢 home lists the scenarios and opens one 392ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.2s
🟢 taps the moving target three times across rebuilds of the list 2.2s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 185ms
🟢 accepts the terms once the button clears the sticky footer 185ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 14.7s
🟢 scrolls the windowed table to the Golden Row and claims it 14.7s
🟢 tests/wishlist.e2e.ts · 2 passed 1.4s
🟢 wishlist › saving toggles the button and the tab badge 193ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.3s

Tested by e2e 0.23.0 · 1m 1s · web · run artifacts

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 87.6k tokens (93% cached) · $0.0038

All 33 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.8s
🟢 control inventory › check sets the checkbox and the radio 1 step 804ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 329ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 942ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.6s
🟢 control inventory › upload attaches the named project files 1 step 2.5s
🟢 control inventory › back returns from the details view to the inventory 1 step 313ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 317ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 423ms
🟢 act signs in with the benchmark account 1 step 423ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 12.1s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 6.2s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.7s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 4.1s
🟢 tests-agent/scenarios.e2e.ts · 21 passed 21 steps · 12 calls 1m 41s
🟢 act completes shadow-dom-form 1 step · 3 calls 4.0s
🟢 act completes lying-labels 1 step · 3 calls 4.3s
🟢 act completes iframe-form 1 step · 3 calls 4.2s
🟢 act completes infinite-scroll 1 step 4.4s
🟢 act completes virtualized-table 1 step 51.5s
🟢 act completes overlay-trap 1 step 911ms
🟢 act completes sticky-chrome 1 step 904ms
🟢 act completes async-states 1 step · 3 calls 6.6s
🟢 act completes debounced-search 1 step 2.4s
🟢 act completes hover-menu 1 step 586ms
🟢 act completes drag-and-drop 1 step 567ms
🟢 act completes kanban-board 1 step 727ms
🟢 act completes context-menu-trap 1 step 2.8s
🟢 act completes stale-dom 1 step 535ms
🟢 act completes otp-auto-advance 1 step 782ms
🟢 act completes date-picker 1 step 933ms
🟢 act completes rich-text-editor 1 step 10.4s
🟢 act completes playbook-cleanup 1 step 638ms
🟢 act completes promo-storefront 1 step 1.3s
🟢 act completes gift-card-purchase 1 step 2.4s
🟢 act filters the list and the URL carries the filter 1 step 257ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 339ms
🟢 act adds an address with a run-unique label 1 step 339ms

Tested by e2e 0.23.0 · 2m 2s · web · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 30 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".changeset/napi-rs-image.md">

<violation number="1" location=".changeset/napi-rs-image.md:2">
P2: These `patch` entries under-version a platform-breaking regression: on Linux ppc64, s390x, and riscv64, mobile and decision now fail during import, while e2e screenshot calls fail. The previous codecs were JavaScript packages; bump these affected 0.x packages to `minor` and start the summary with `**Breaking:**`. 

(Based on your team's feedback about 0.x changeset bumps.) .</violation>
</file>

<file name="packages/decision/src/png.ts">

<violation number="1" location="packages/decision/src/png.ts:26">
P2: This pixel-layout conversion duplicates the corresponding helpers in `packages/mobile/src/png.ts` and `packages/e2e/src/internal/image.ts`. Move the shared conversion into a common internal helper so format and rounding fixes cannot drift between packages.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/decision/tests/overlay.test.ts Resolved
@@ -0,0 +1,68 @@
import { endianness } from 'node:os';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This pixel-layout conversion duplicates the corresponding helpers in packages/mobile/src/png.ts and packages/e2e/src/internal/image.ts. Move the shared conversion into a common internal helper so format and rounding fixes cannot drift between packages.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/decision/src/png.ts, line 26:

<comment>This pixel-layout conversion duplicates the corresponding helpers in `packages/mobile/src/png.ts` and `packages/e2e/src/internal/image.ts`. Move the shared conversion into a common internal helper so format and rounding fixes cannot drift between packages.</comment>

<file context>
@@ -0,0 +1,68 @@
+ * widened by the addon: opacity 1 keeps every sample and adds the opaque
+ * alpha channel, several times faster than `spread` does it.
+ */
+export function decodeRgba(bytes: Uint8Array): Rgba {
+  const decoder = new Transformer(bytes);
+  const { width, height, format, colorType } = decoder.metadataSync();
</file context>

@kdy1 kdy1 Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude-sonnet-5-5] responding on behalf of kdy1

Left as three copies, and left open for the owner. @e2e-dev/mobile and @e2e-dev/decision reach e2e only through its public entries (AGENTS.md, Layout), so one shared helper means a new export on e2e or e2e/engine, or a new package. That is a public-contract and release decision, not part of a codec swap. readPngSize is already copied in web and mobile for the same reason. It is listed under the PR body decisions.

Comment thread packages/mobile/tests/helpers/png.ts Resolved
Comment on lines +2 to +7
"e2e": patch
"@e2e-dev/decision": patch
"@e2e-dev/mobile": patch
---

PNG decoding and encoding now run on `@napi-rs/image`, a native addon with a prebuilt binary per platform, in place of `fast-png` and `pngjs`: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a `toHaveScreenshot` capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. `e2e` loads it when the first screenshot is read or written, so `--version` and a run that takes none never need it, and a missing binding fails that call with `IMAGE_CODEC_UNAVAILABLE`; `@e2e-dev/mobile` and `@e2e-dev/decision` load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: These patch entries under-version a platform-breaking regression: on Linux ppc64, s390x, and riscv64, mobile and decision now fail during import, while e2e screenshot calls fail. The previous codecs were JavaScript packages; bump these affected 0.x packages to minor and start the summary with **Breaking:**.

(Based on your team's feedback about 0.x changeset bumps.) .

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .changeset/napi-rs-image.md, line 2:

<comment>These `patch` entries under-version a platform-breaking regression: on Linux ppc64, s390x, and riscv64, mobile and decision now fail during import, while e2e screenshot calls fail. The previous codecs were JavaScript packages; bump these affected 0.x packages to `minor` and start the summary with `**Breaking:**`. 

(Based on your team's feedback about 0.x changeset bumps.) .</comment>

<file context>
@@ -0,0 +1,7 @@
+---
+"e2e": patch
+"@e2e-dev/decision": patch
+"@e2e-dev/mobile": patch
</file context>
Suggested change
"e2e": patch
"@e2e-dev/decision": patch
"@e2e-dev/mobile": patch
---
PNG decoding and encoding now run on `@napi-rs/image`, a native addon with a prebuilt binary per platform, in place of `fast-png` and `pngjs`: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a `toHaveScreenshot` capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. `e2e` loads it when the first screenshot is read or written, so `--version` and a run that takes none never need it, and a missing binding fails that call with `IMAGE_CODEC_UNAVAILABLE`; `@e2e-dev/mobile` and `@e2e-dev/decision` load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.
"e2e": minor
"@e2e-dev/decision": minor
"@e2e-dev/mobile": minor
---
**Breaking:** PNG decoding and encoding now run on `@napi-rs/image`, a native addon with a prebuilt binary per platform, in place of `fast-png` and `pngjs`: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a `toHaveScreenshot` capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. `e2e` loads it when the first screenshot is read or written, so `--version` and a run that takes none never need it, and a missing binding fails that call with `IMAGE_CODEC_UNAVAILABLE`; `@e2e-dev/mobile` and `@e2e-dev/decision` load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.

@kdy1 kdy1 Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude-sonnet-5-5] responding on behalf of kdy1

Left open for the owner. AGENTS.md says never pick minor on our own, and whether losing Linux on ppc64, s390x, and riscv64 is a breaking change is a release call. It is the first item under "Decisions for the reviewer" in the PR body, and the changeset names the missing platforms.

Comment thread docs/reference/errors.mdx Resolved
Comment thread packages/e2e/tests/unit/image.test.ts Resolved
… helpers

- image.test.ts gains a 2-bit gray row and decision's overlay test a palette
  row, so every color type decodePng claims is exercised.
- The six test helpers that decode with Transformer now throw unless the
  bytes are a PNG, as pngjs did, so a JPEG cannot pass for one.
- The IMAGE_CODEC_UNAVAILABLE row in errors.mdx names the agent screenshots
  that need the addon: those the runner resizes, over 768 px.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 12 replayed from cache · 58 model calls · 368.0k tokens (82% cached) · $0.0247

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 58 calls 8m 45s
🟢 act completes Login Form 1 step 40.0s
🟢 act completes Infinite Scroll List 1 step 30.6s
🟢 act completes Modal Flow 1 step 10.8s
🟢 act completes Bottom Tabs 1 step 7.6s
🟢 act completes Text Input Variations 1 step 14.3s
🟢 act completes Broken Accessibility 2 steps · 8 calls 44.2s
🟢 act completes Vision Only 2 steps · 6 calls 28.6s
🟢 act completes Gestures 1 step · 6 calls 28.0s
🟢 act completes Async States 1 step · 9 calls 33.9s
🟢 act completes Debounced Search 1 step · 1 call 27.4s
🟢 act completes Flattened Registration Form 2 steps · 1 call 29.2s
🟢 act completes Flattened Login 2 steps · 11 calls 1m 3s
🟢 act completes Sticky Chrome Target 1 step 10.8s
🟢 act completes WebView Accessibility 1 step 12.2s
🟢 act completes Permission Prompt 1 step 11.0s
🟢 act completes Photo Picker 1 step · 2 calls 31.6s
🟢 act completes Product Catalog 1 step 10.4s
🟢 act completes Error Recovery 1 step 9.9s
🟢 act completes Choice Controls 1 step · 6 calls 36.7s
🟢 act completes Stripe PaymentSheet 1 step · 8 calls 33.0s
🟢 act completes Apple Pay 1 step 12.3s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.0 · 8m 46s · ios-simulator · run artifacts

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 1 flaky, 19 passed

24 agent steps · 15 replayed from cache · 39 model calls · 285.7k tokens (84% cached) · $0.0187

⚠️ 1 flaky test passed on a retry

⚠️ act completes Gestures
tests-agent/scenarios.e2e.ts:266

ACTION_FAILED at step 4 of 4: agent.act "Complete this scenario as the screen instructs: long-press the first target, swipe the card left, then double-tap the last target", after 26.1s and 11 model calls

agent.act failed: The cached long-press is complete, but repeated right-to-left swipes across the visible “Swipe me left” card left the screen on Step 2 of 3. I could not complete the swipe or reach the final target.

  • Turn 8: swipe({"from":{"x":220,"y":460},"to":{"x":20,"y":460}}) → [swipe] Swiped from (220, 460) to (20, 460).
  • Turn 9: screenshot({}) → [screenshot] The screen changed substantially since revision b12. Current screen (revision b14, path dev.e2e.benchmark, 18 nodes):
  • Turn 10: drag_at({"x":170,"y":460,"to_x":25,"to_y":460}) → [drag_at] drag_at (170, 460) to (25, 460) failed: UNSUPPORTED_CAPABILITY: the drag from (531, 1436) to (78, 1436) does not join two different controls the scre…
  • Screen: dev.e2e.benchmark

Evidence: screenshot, log

All 20 tests in 1 file
Test Agent Time
⚠️ tests-agent/scenarios.e2e.ts · 1 flaky, 19 passed 24 steps · 39 calls 4m 48s
🟢 act completes Login Form 1 step 22.4s
🟢 act completes Infinite Scroll List 1 step 21.3s
🟢 act completes Modal Flow 1 step 9.8s
🟢 act completes Bottom Tabs 1 step 6.9s
🟢 act completes Text Input Variations 1 step 6.1s
🟢 act completes Broken Accessibility 2 steps · 5 calls 14.2s
🟢 act completes Vision Only 2 steps · 1 call 6.5s
⚠️ act completes Gestures (1 failed attempt first) 1 step · 9 calls 22.4s
🟢 act completes Async States 1 step · 13 calls 26.1s
🟢 act completes Debounced Search 1 step 6.7s
🟢 act completes Huge Virtualized List 1 step 1m 1s
🟢 act completes Flattened Registration Form 2 steps · 1 call 7.8s
🟢 act completes Flattened Login 2 steps · 6 calls 14.6s
🟢 act completes Sticky Chrome Target 1 step 7.3s
🟢 act completes WebView Accessibility 1 step · 4 calls 13.4s
🟢 act completes Permission Prompt 1 step 6.3s
🟢 act completes Product Catalog 1 step 8.1s
🟢 act completes Error Recovery 1 step 8.8s
🟢 act completes Choice Controls 1 step 8.3s
🟢 act completes Sequential Onboarding 1 step 9.5s

Tested by e2e 0.23.0 · 5m 18s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🟢 e2e android: 57 passed, 1 skipped

All 58 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 46s
🟢 control inventory › a label query resolves a labeled text field 23.6s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 11.8s
🟢 control inventory › a placeholder query and attribute reads find the field 5.6s
🟢 control inventory › a secure field takes a fill and denies value reads 6.7s
🟢 control inventory › press sends a character and Enter through the soft keyboard 9.5s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 7.8s
🟢 control inventory › toBeFocused follows the focus 7.9s
🟢 control inventory › a switch checks, unchecks, and reports its state 8.3s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 7.1s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 10.4s
🟢 control inventory › a tab role answers getByRole 3.6s
🟢 control inventory › a radio group reports the checked option 7.6s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.7s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.4s
🟢 control inventory › a header answers its accessible name 3.8s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.4s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.2s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.1s
🟢 control inventory › a double tap lands inside the 300 ms window 6.6s
🟢 control inventory › an ambiguous locator fails at once 4.9s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 7.1s
🟢 refusals › verbs a device never declares are refused before the engine 3ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.6s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 13s
🟢 device fixture › setOrientation rotates the window the app measures 4.9s
🟢 device fixture › home leaves the app, openApp brings it back where it was 4.1s
🟢 device fixture › foregroundApp sees the home screen after home() 4.0s
🟢 device fixture › closeApp then openApp relaunches on the home list 6.7s
🟢 device fixture › closeApp leaves the next commands on the same device 6.3s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 4.0s
🟢 device fixture › device.back pops the scenario 4.4s
🟢 device fixture › unstable_raw reaches the test session through agent-device 3.1s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 15.3s
🟢 device fixture › setLocation and clearLocation change what the app reads 20.0s
🟢 tests/device.e2e.ts · 5 passed 37.0s
🟢 permission prompt › allowing the system dialog enables the microphone 5.2s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 11.5s
🟢 sequential onboarding echoes the exact values 9.3s
🟢 onboarding rejects a malformed email before advancing 5.1s
🟢 dismissing the keyboard over choice controls leaves them as they were 6.0s
🟢 tests/flows-lists.e2e.ts · 7 passed 1m 47s
🟢 infinite scroll reaches the target item 14.5s
🟢 product catalog adds exactly two of the right variant 8.1s
🟢 gestures: long-press, then swipe left 7.8s
🟢 gestures: double-tap completes the flow 9.0s
🟢 sticky chrome: accept below the fold, then continue 8.1s
🟢 async states: load, pull to refresh, claim through the toast 11.2s
🟢 huge virtualized list reaches row 512 47.9s
🟢 tests/flows.e2e.ts · 7 passed 46.3s
🟢 text inputs unlock the submit button 5.4s
🟢 filled text inputs report their values 5.0s
🟢 modal flow confirms through the native alert 7.8s
🟢 bottom tabs act inside the last tab 6.0s
🟢 error recovery retries, then confirms the delete 8.2s
🟢 choice controls place the exact order 5.6s
🟢 debounced search selects the target once results arrive 8.3s
🟢 tests/login-form.e2e.ts · 3 passed 25.5s
🟢 login form › rejects a malformed email 6.7s
🟢 login form › rejects an unknown account 7.0s
🟢 login form › signs in and logs out 11.8s
🟢 tests/smoke.e2e.ts · 1 passed 4.9s
🟢 home lists the scenarios and opens one 4.9s

Tested by e2e 0.23.0 · 7m 39s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 3m 19s
🟢 control inventory › a label query resolves a labeled text field 15.4s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 18.1s
🟢 control inventory › a placeholder query and attribute reads find the field 5.3s
🟢 control inventory › a secure field takes a fill and denies value reads 10.5s
🟢 control inventory › press sends a character and Enter through the soft keyboard 10.7s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 12.2s
🟢 control inventory › toBeFocused follows the focus 10.2s
🟢 control inventory › a switch checks, unchecks, and reports its state 12.0s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 9.5s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 14.7s
🟢 control inventory › a radio group reports the checked option 9.2s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 8.3s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.2s
🟢 control inventory › a header answers its accessible name 4.2s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.1s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.6s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.3s
🟢 control inventory › pointer verbs land at the point asked for 10.2s
🟢 control inventory › an ambiguous locator fails at once 6.0s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 12.4s
🟢 refusals › verbs a device never declares are refused before the engine 3ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 494ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 57s
🟢 device fixture › setAppearance flips the color scheme the app reads 18.1s
🟢 device fixture › setOrientation rotates the window the app measures 11.6s
🟢 device fixture › the clipboard reads back what was written 8.8s
🟢 device fixture › home leaves the app, openApp brings it back where it was 8.3s
🟢 device fixture › closeApp then openApp relaunches on the home list 9.4s
🟢 device fixture › closeApp leaves the next commands on the same device 13.1s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 8.3s
🟢 device fixture › device.back pops the scenario 6.7s
🟢 device fixture › unstable_raw reaches the test session through agent-device 5.5s
🟢 device fixture › setLocation and clearLocation change what the app reads 15.1s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 12.2s
🟢 tests/device.e2e.ts · 3 passed 1m 8s
🟢 sequential onboarding echoes the exact values 41.2s
🟢 onboarding rejects a malformed email before advancing 14.1s
🟢 dismissing the keyboard over choice controls leaves them as they were 12.4s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 19s
🟢 infinite scroll reaches the target item 24.4s
🟢 product catalog adds exactly two of the right variant 9.9s
🟢 gestures: long-press, then swipe left 9.0s
🟢 sticky chrome: accept below the fold, then continue 10.1s
🟢 async states: load, pull to refresh, claim through the toast 16.1s
🟢 huge virtualized list reaches row 512 1m 57s
🟢 web view coupon form applies the code shown on the page 12.8s
🟢 tests/flows.e2e.ts · 7 passed 1m 9s
🟢 text inputs unlock the submit button 13.2s
🟢 filled text inputs report their values 10.3s
🟢 modal flow confirms through the native alert 9.7s
🟢 bottom tabs act inside the last tab 7.5s
🟢 error recovery retries, then confirms the delete 9.3s
🟢 choice controls place the exact order 7.5s
🟢 debounced search selects the target once results arrive 11.3s
🟢 tests/login-form.e2e.ts · 3 passed 49.8s
🟢 login form › rejects a malformed email 13.8s
🟢 login form › rejects an unknown account 14.2s
🟢 login form › signs in and logs out 21.8s
🟢 tests/smoke.e2e.ts · 1 passed 8.8s
🟢 home lists the scenarios and opens one 8.8s

Tested by e2e 0.23.0 · 6m 25s · ios-simulator · run artifacts

@kdy1

kdy1 commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

[claude-sonnet-5-5] responding on behalf of kdy1

Left for you:

  • Version bump (open thread on .changeset/napi-rs-image.md). The changeset is a patch. Linux on ppc64, s390x, and riscv64 get no @napi-rs/image binding, so screenshots fail there and @e2e-dev/mobile and @e2e-dev/decision fail at import. Whether that is a **Breaking:** minor is a release call.
  • Shared layout code (open thread on packages/decision/src/png.ts). The pixel-layout step has three copies, because sharing it means a new export on e2e or e2e/engine, or a new package.
  • Encoded size. FilterType.Adaptive makes a Chromium capture about 15% larger than fast-png wrote it. The "smaller of no filter and adaptive" option is in the PR body and is not done.
  • Not verified: Windows, and the agentic suites live against a model (CI replays their recordings).

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! I think its fine to ship it as patch no need for a minor bump. good job!

@okwasniewski
okwasniewski merged commit 34ccc5f into main Nov 6, 2026
32 checks passed
@okwasniewski
okwasniewski deleted the kdy1/replace-png-libraries branch November 6, 2026 09:20
@github-actions github-actions Bot mentioned this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:L Up to 499 changed lines, not counting tests, fixtures, translated docs, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants