fix(#4852): resolve OutOfMemoryError for large files in Cache.java by volodya-lombrozo · Pull Request #4869 · objectionary/eo

volodya-lombrozo · 2026-02-09T10:27:43Z

This PR resolves the OutOfMemoryError issue in Cache.java when handling large files.

Fixes #4852

Summary by CodeRabbit

Tests
- Added test cases to verify SHA-256 hash computation correctness for files of varying sizes.

…h calculation

coderabbitai · 2026-02-09T10:28:06Z

📝 Walkthrough

Walkthrough

The Cache.sha() method is refactored to stream files in chunks rather than loading entire files into memory, addressing an OutOfMemoryError issue with large files. Two test cases are added to verify hash correctness for both large and small files.

Changes

Cohort / File(s)	Summary
Hash Streaming Optimization `eo-maven-plugin/src/main/java/org/eolang/maven/Cache.java`	Replaced in-memory file read with buffered streaming via try-with-resources; digest computed incrementally in chunks before final digest() call to prevent memory exhaustion.
Hash Verification Tests `eo-maven-plugin/src/test/java/org/eolang/maven/CacheTest.java`	Added two test methods: `generatesCorrectHashForLargeFile()` (100,000 lines) and `generatesCorrectHashForTinyFile()` ("x") to verify Base64-encoded SHA-256 output correctness.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

feat(#4846): implement a simple caching mechanism in the Maven plugin #4847: Directly modifies the same Cache.sha(Path) method for Base64 SHA-256 computation.
test(#4853): add comprehensive tests for Cache class in CacheTest.java #4854: Adds test cases (e.g., writesCorrectShaHash) that verify the Base64-encoded SHA-256 output of the Cache class.
fix(#4863): resolve failed tests in MjLintTest.java related to caching #4864: Refactors related caching flows to use the Cache/CachePath-based mechanism affected by this change.

Suggested reviewers

yegor256

Poem

🐰 A file too large made memory weep,
With buffers full, no room to keep,
But streaming chunks, we hop with glee,
Through bytes in sequences, wild and free,
No OutOfMemory error we shall see! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main fix: resolving OutOfMemoryError for large files in Cache.java, directly matching the changeset's streaming approach implementation.
Linked Issues check	✅ Passed	The PR successfully implements the streaming SHA-256 hash calculation to resolve OutOfMemoryError and includes test cases for both large and small files, meeting issue `#4852` requirements.
Out of Scope Changes check	✅ Passed	All changes are directly related to fixing the OutOfMemoryError in Cache.java through streaming implementation and corresponding tests; no out-of-scope modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments

eo-maven-plugin/src/test/java/org/eolang/maven/CacheTest.java (1)

163-193: Test validates correctness but doesn't exercise the OOM scenario.

The content string (~1.3 MB) is held entirely in memory both in the test and in the p -> content lambda. This test confirms hash correctness for a multi-line file but won't catch an OOM regression since the file is far too small to trigger one. Consider adding a comment clarifying the intent is correctness validation, or (if feasible in CI) testing with a file large enough to exceed a constrained -Xmx.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-02-09T10:30:00Z

🚀 Performance Analysis

All benchmarks are within the acceptable range. No critical degradation detected (threshold is 100%). Please refer to the detailed report for more information.

Click to see the detailed report

Test	Base Score	PR Score	Change	% Change	Unit	Mode
`benchmarks.XmirBench.xmirToEO`	205.086	170.512	-34.574	-16.86%	ms/op	Average Time

✅ Performance gain: benchmarks.XmirBench.xmirToEO is faster by 34.574 ms/op (16.86%)

Copilot

Pull request overview

This PR addresses OutOfMemoryError risk in Cache hashing by switching SHA-256 computation from loading full file bytes into memory to a streaming approach, and updates tests around SHA generation.

Changes:

Replace Files.readAllBytes(...) hashing with buffered streaming hashing in Cache.sha(...).
Remove the related puzzle/todo from Cache.java.
Add new tests that assert SHA-256 hash file contents for “large” and tiny inputs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
eo-maven-plugin/src/main/java/org/eolang/maven/Cache.java	Streams file content into `MessageDigest` to avoid hashing large files via `readAllBytes`.
eo-maven-plugin/src/test/java/org/eolang/maven/CacheTest.java	Adds additional SHA-256 verification tests for larger and tiny content sizes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-09T10:31:29Z

+        final int lines = 100_000;
+        final StringBuilder builder = new StringBuilder(lines * 10);
+        IntStream.range(0, lines).forEach(
+            i -> builder.append("Line ").append(i).append('\n')
+        );
+        final String content = builder.toString();
+        Files.writeString(source, content, StandardCharsets.UTF_8);


The "large file" test still builds the entire file content in memory (StringBuilder + content String + content.getBytes for expected digest). That makes the test itself memory-heavy and it won’t catch a regression back to Files.readAllBytes (the old implementation would still pass at this size). Consider generating/writing the file incrementally and computing the expected SHA-256 via a streaming digest as well, so the test stays low-memory and can be scaled without holding the full content in RAM.

Copilot · 2026-02-09T10:31:30Z

+    @Test
+    void generatesCorrectHashForTinyFile(
+        @Mktmp final Path temp
+    ) throws IOException, NoSuchAlgorithmException {
+        final var cache = temp.resolve("cache");
+        Files.createDirectories(cache);
+        final var source = temp.resolve("tiny.txt");
+        final String content = "x";
+        Files.writeString(source, content, StandardCharsets.UTF_8);
+        final var target = temp.resolve("out.txt");
+        final var tail = source.getFileName();
+        new Cache(cache, p -> content).apply(source, target, tail);
+        MatcherAssert.assertThat(
+            "SHA-256 hash file has incorrect content for tiny file",
+            Files.readString(
+                cache.resolve(String.format("%s.sha256", tail)),
+                StandardCharsets.UTF_8
+            ),
+            Matchers.equalTo(
+                Base64.getEncoder().encodeToString(
+                    MessageDigest.getInstance("SHA-256")
+                        .digest(content.getBytes(StandardCharsets.UTF_8))
+                )
+            )
+        );


The new tiny-file hash test largely overlaps with the existing writesCorrectShaHash() coverage (both validate SHA-256 file contents for a small input). Unless it’s guarding a specific edge case, consider removing it or turning these into a single parameterized test to avoid duplicated setup/logic.

volodya-lombrozo · 2026-02-09T12:04:39Z

@yegor256 could you have a look, please?

volodya-lombrozo · 2026-02-10T06:23:07Z

@yegor256 friendly reminder

0crat · 2026-02-10T18:17:00Z

@volodya-lombrozo Thanks for the contribution! You've earned +16 points for your code submission. This reward follows our team policy which awards 16 points for reviews provided, plus additional points based on your hits-of-code contribution. Please keep them coming! Your running score is +446; don't forget to check your Zerocracy account too.

feat(objectionary#4852): Implement streaming approach for SHA-256 has…

bdde463

…h calculation

Copilot AI review requested due to automatic review settings February 9, 2026 10:27

Copilot started reviewing on behalf of volodya-lombrozo February 9, 2026 10:28 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

volodya-lombrozo requested a review from yegor256 February 9, 2026 12:04

yegor256 merged commit b77bf49 into objectionary:master Feb 10, 2026
31 of 32 checks passed

coderabbitai Bot mentioned this pull request Apr 10, 2026

feat(#4983): implement SHA-256 hashing for directories in Cache #4984

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(#4852): resolve OutOfMemoryError for large files in Cache.java#4869

fix(#4852): resolve OutOfMemoryError for large files in Cache.java#4869
yegor256 merged 1 commit into
objectionary:masterfrom
volodya-lombrozo:4852-outofmemory-fix

volodya-lombrozo commented Feb 9, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Feb 9, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

Uh oh!

github-actions Bot commented Feb 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

volodya-lombrozo commented Feb 9, 2026

Uh oh!

volodya-lombrozo commented Feb 10, 2026

Uh oh!

Uh oh!

0crat commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

volodya-lombrozo commented Feb 9, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

Uh oh!

github-actions Bot commented Feb 9, 2026

🚀 Performance Analysis

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

volodya-lombrozo commented Feb 9, 2026

Uh oh!

volodya-lombrozo commented Feb 10, 2026

Uh oh!

Uh oh!

0crat commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

volodya-lombrozo commented Feb 9, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Feb 9, 2026 •

edited

Loading