Skip to content

Stream SIRI-ET initial data HTTP response directly into XML parser#7501

Merged
vpaturet merged 1 commit intoopentripplanner:dev-2.xfrom
entur:stream-siri-et-initial-data-parse
Apr 13, 2026
Merged

Stream SIRI-ET initial data HTTP response directly into XML parser#7501
vpaturet merged 1 commit intoopentripplanner:dev-2.xfrom
entur:stream-siri-et-initial-data-parse

Conversation

@vpaturet
Copy link
Copy Markdown
Contributor

@vpaturet vpaturet commented Apr 7, 2026

Summary

Stream the SIRI-ET PubSub initial data HTTP response directly into the StAX/JAXB XML parser instead of buffering the entire payload into a ByteString.

This eliminates a ~325 MB (TST) / ~580 MB (PRD) short-lived allocation that contributes to GC pressure during startup. The SiriXml.parseXml(InputStream) parser already supports streaming via XMLStreamReader, and this pattern is already used by SiriAzureUpdater and SiriFmDataSource.

Before

HTTP response → ByteString (full buffer in memory) → SiriXml.parseXml()

After

HTTP response → SiriXml.parseXml(response.body())  (streamed directly)

Unit tests

No unit tests, tested manually.

Documentation

No documentation changes needed — no new configuration options or public API changes.

Instead of buffering the entire HTTP response (~325-580 MB) into a
ByteString before parsing, stream the response body directly into
SiriXml.parseXml() via StAX. This eliminates a large short-lived
allocation that contributes to GC pressure during startup.

The streaming pattern is already used by SiriAzureUpdater and
SiriFmDataSource.
@vpaturet vpaturet added !Optimization The feature is to improve performance. !Technical Debt Improve code quality, no functional changes. +Real-Time The issue/PR is related to RealTime updates +Skip Changelog This is not a relevant change for a product owner since last release. labels Apr 7, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 7, 2026

Codecov Report

❌ Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.03%. Comparing base (1158ac0) to head (57914e8).
⚠️ Report is 16 commits behind head on dev-2.x.

Files with missing lines Patch % Lines
...r/google/GooglePubsubEstimatedTimetableSource.java 0.00% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             dev-2.x    #7501   +/-   ##
==========================================
  Coverage      71.02%   71.03%           
- Complexity     21002    21006    +4     
==========================================
  Files           2352     2352           
  Lines          87266    87265    -1     
  Branches        8635     8635           
==========================================
+ Hits           61983    61986    +3     
+ Misses         22293    22292    -1     
+ Partials        2990     2987    -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vpaturet vpaturet marked this pull request as ready for review April 7, 2026 11:20
@vpaturet vpaturet requested a review from a team as a code owner April 7, 2026 11:20
@vpaturet vpaturet added the Entur Test This is currently being tested at Entur label Apr 7, 2026
@vpaturet vpaturet added this pull request to the merge queue Apr 13, 2026
Merged via the queue into opentripplanner:dev-2.x with commit 20888a4 Apr 13, 2026
12 checks passed
@vpaturet vpaturet deleted the stream-siri-et-initial-data-parse branch April 13, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Entur Test This is currently being tested at Entur !Optimization The feature is to improve performance. +Real-Time The issue/PR is related to RealTime updates +Skip Changelog This is not a relevant change for a product owner since last release. !Technical Debt Improve code quality, no functional changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants