Skip to content

fix: reduce peak memory ~9× in LZW decoder and TIFF encoder; clean up TODO#104

Merged
Hexagon merged 2 commits intomainfrom
copilot/remove-finished-sections-todo
Apr 9, 2026
Merged

fix: reduce peak memory ~9× in LZW decoder and TIFF encoder; clean up TODO#104
Hexagon merged 2 commits intomainfrom
copilot/remove-finished-sections-todo

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 9, 2026

Two medium-severity memory inefficiencies where large typed data was funnelled through number[] (8 bytes/element), causing ~9× peak memory overhead before final Uint8Array conversion.

Changes

src/utils/lzw.ts — M5

  • LZWDecoder.decompress() replaced number[] accumulator with a doubling Uint8Array buffer
  • Entries appended via buf.set(entry, len) (bulk) instead of byte-by-byte push()
  • Returns buf.slice(0, len) to release the oversized backing buffer and allow GC

src/formats/tiff.ts — M6

  • Both encode() and encodeFrames() now keep compressed pixel frames as Uint8Array chunks; only the small IFD section (~hundreds of bytes) is built in a number[]
  • Final output assembled with a single pre-allocated Uint8Array + set() calls — eliminates the intermediate JS number array for pixel data entirely
// Before: byte-by-byte copy of entire compressed frame into number[]
for (let i = 0; i < pixelData.length; i++) result.push(pixelData[i]);

// After: direct set into pre-sized output buffer
out.set(frame, pixelPos);

TODO.md

Removed all verified-complete items (C1–C3, H1–H4, M1–M4, M7, L5). Remaining open: M5/M6 (now addressed above) and L1–L4, L6–L8.

CHANGELOG.md

Added entries for M5 and M6 under [Unreleased].

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces peak memory usage in two hot paths by eliminating large intermediate number[] byte accumulators, replacing them with Uint8Array-based buffering and final assembly via pre-sized typed arrays. This aligns with the project’s focus on efficient binary image encode/decode paths.

Changes:

  • Refactored LZWDecoder.decompress() to write into a growable Uint8Array instead of a number[].
  • Refactored TIFF encode() / encodeFrames() to keep compressed pixel data as Uint8Array chunks and assemble output via a single preallocated Uint8Array.
  • Cleaned up TODO.md and added [Unreleased] changelog entries for the memory improvements.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
TODO.md Removes completed items and marks M5/M6 as fixed.
src/utils/lzw.ts Implements a growable Uint8Array output buffer for LZW decompression.
src/formats/tiff.ts Avoids pixel data copying into number[]; assembles final TIFF bytes with Uint8Array.set().
CHANGELOG.md Documents the LZW/TIFF memory improvements under [Unreleased].

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CHANGELOG.md Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@Hexagon Hexagon merged commit 71e36f3 into main Apr 9, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants