Repository navigation
perf: decode and encode PNGs with @napi-rs/image #1587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| "e2e": patch | ||
| "@e2e-dev/decision": patch | ||
| "@e2e-dev/mobile": patch | ||
| --- | ||
|
|
||
| PNG decoding and encoding now run on `@napi-rs/image`, a native addon with a prebuilt binary per platform, in place of `fast-png` and `pngjs`: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a `toHaveScreenshot` capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. `e2e` loads it when the first screenshot is read or written, so `--version` and a run that takes none never need it, and a missing binding fails that call with `IMAGE_CODEC_UNAVAILABLE`; `@e2e-dev/mobile` and `@e2e-dev/decision` load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| import { endianness } from 'node:os'; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: This pixel-layout conversion duplicates the corresponding helpers in Prompt for AI agents
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [claude-sonnet-5-5] responding on behalf of kdy1 Left as three copies, and left open for the owner. |
||
| import { FilterType, JsColorType, Transformer } from '@napi-rs/image'; | ||
| /** An 8-bit RGBA image, rows top to bottom. */ | ||
| export interface Rgba { | ||
| readonly width: number; | ||
| readonly height: number; | ||
| readonly data: Uint8Array; | ||
| } | ||
| const LITTLE_ENDIAN = endianness() === 'LE'; | ||
| /** Channels per pixel, and whether each is a 16-bit sample, for every color type a decode reads: 8-bit RGB is widened by the addon before that. */ | ||
| const LAYOUTS: Readonly<Partial<Record<JsColorType, { readonly channels: number; readonly wide: boolean }>>> = { | ||
| [JsColorType.L8]: { channels: 1, wide: false }, | ||
| [JsColorType.La8]: { channels: 2, wide: false }, | ||
| [JsColorType.Rgba8]: { channels: 4, wide: false }, | ||
| [JsColorType.L16]: { channels: 1, wide: true }, | ||
| [JsColorType.La16]: { channels: 2, wide: true }, | ||
| [JsColorType.Rgb16]: { channels: 3, wide: true }, | ||
| [JsColorType.Rgba16]: { channels: 4, wide: true }, | ||
| }; | ||
| /** | ||
| * Decodes PNG bytes to 8-bit RGBA, whatever the color type, bit depth, or | ||
| * palette; throws on bytes that are not a PNG. An 8-bit RGB screenshot is | ||
| * widened by the addon: opacity 1 keeps every sample and adds the opaque | ||
| * alpha channel, several times faster than `spread` does it. | ||
| */ | ||
| export function decodeRgba(bytes: Uint8Array): Rgba { | ||
| const decoder = new Transformer(bytes); | ||
| const { width, height, format, colorType } = decoder.metadataSync(); | ||
| if (format !== 'png') throw new Error(`not a PNG: the bytes are ${format}`); | ||
| const rgb = colorType === JsColorType.Rgb8; | ||
| const raw = (rgb ? decoder.opacity(1) : decoder).rawPixelsSync(); | ||
| const layout = LAYOUTS[rgb ? JsColorType.Rgba8 : colorType]; | ||
| if (layout === undefined) throw new Error(`unsupported PNG color type ${colorType}`); | ||
| const { channels, wide } = layout; | ||
| if (raw.length !== width * height * channels * (wide ? 2 : 1)) throw new Error('the decoded PNG pixels do not fill its size'); | ||
| const samples = new Uint8Array(raw.buffer, raw.byteOffset, raw.byteLength); | ||
| if (channels === 4 && !wide) return { width, height, data: samples }; | ||
| return { width, height, data: spread(wide ? narrow(samples) : samples, channels, width * height) }; | ||
| } | ||
| /** | ||
| * Encodes an image as PNG bytes, each row adaptively filtered: the addon's | ||
| * default writes no filter, which makes screenshots markedly larger. | ||
| */ | ||
| export function encodeRgba(image: Rgba): Uint8Array { | ||
| const png = Transformer.fromRgbaPixels(image.data, image.width, image.height).pngSync({ filterType: FilterType.Adaptive }); | ||
| return new Uint8Array(png.buffer, png.byteOffset, png.byteLength); | ||
| } | ||
| /** 16-bit samples in native byte order as 8-bit samples, rounded. */ | ||
| function narrow(bytes: Uint8Array): Uint8Array { | ||
| const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); | ||
| const samples = new Uint8Array(bytes.length / 2); | ||
| for (let at = 0; at < samples.length; at += 1) samples[at] = Math.round((view.getUint16(at * 2, LITTLE_ENDIAN) * 255) / 65535); | ||
| return samples; | ||
| } | ||
| /** 8-bit samples of `channels` per pixel (gray, gray and alpha, RGB) as RGBA; a gray or RGB image is opaque. */ | ||
| function spread(samples: Uint8Array, channels: number, pixels: number): Uint8Array { | ||
| const out = new Uint8Array(pixels * 4); | ||
| const gray = channels <= 2; | ||
| const alpha = channels === 2 || channels === 4; | ||
| for (let pixel = 0, from = 0, to = 0; pixel < pixels; pixel += 1, from += channels, to += 4) { | ||
| const r = samples[from]!; | ||
| out[to] = r; | ||
| out[to + 1] = gray ? r : samples[from + 1]!; | ||
| out[to + 2] = gray ? r : samples[from + 2]!; | ||
| out[to + 3] = alpha ? samples[from + channels - 1]! : 255; | ||
| } | ||
| return out; | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: These
patchentries under-version a platform-breaking regression: on Linux ppc64, s390x, and riscv64, mobile and decision now fail during import, while e2e screenshot calls fail. The previous codecs were JavaScript packages; bump these affected 0.x packages tominorand start the summary with**Breaking:**.(Based on your team's feedback about 0.x changeset bumps.) .
Prompt for AI agents
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[claude-sonnet-5-5] responding on behalf of kdy1
Left open for the owner. AGENTS.md says never pick
minoron our own, and whether losing Linux on ppc64, s390x, and riscv64 is a breaking change is a release call. It is the first item under "Decisions for the reviewer" in the PR body, and the changeset names the missing platforms.