Fix find_fragment timestamp predicate and add property-based tests

[lukebakken]

Summary

Fixes a bug in find_fragment that caused timestamp subscriptions to start from the wrong position, fixes several other bugs in the remote tier read path, and adds comprehensive property-based tests.

Bug fix: find_fragment timestamp predicate

find_fragment used Ts >= FTs to select the manifest fragment for a timestamp subscription. When Ts falls between two fragments — after fragment N's LTs but before fragment N+1's FTs — the predicate incorrectly returned fragment N, causing the reader to deliver the wrong message.

The correct predicate is Ts >= LTs: find the first fragment whose last timestamp is at or after Ts. The read_from_remote_tier_by_timestamp integration test is also fixed to capture Timestamp3 before publishing offset 2.

Other fixes

• Fix range_spec_to_location_number/2: suffix range used addition instead of subtraction; open-ended byte range returned infinity instead of the remaining file size
• Fix init_offset_reader/2 calling osiris_log directly for the local path; route through init_local_reader/2 instead
• Preserve ?C_OSIRIS_LOG_FIRST_OFFSET after local segment deletion; only update the shared atomic used for local/remote tier routing

Tests

• Add integration tests for reading from the remote tier by offset and timestamp
• Add property-based tests for range_spec_to_location_number/2, find_fragment timestamp behaviour, find_index_position offset and timestamp specs, and rabbitmq_stream_s3_array (partition_point, binary_search_by, rfind, fold)
• Add diagnostic logging to surface manifest entry timestamps in CI output
• Upload CT logs as an artifact on test failure

[lukebakken]

AI note on the find_fragment predicate change for the reviewer's benefit.

The fix changes the timestamp predicate from Ts >= FTs to Ts >= LTs. This only affects behaviour when there is a gap between fragments — i.e. LTs(N) < FTs(N+1) — which happens when there is a period of inactivity between segment rolls.

For contiguous fragments (LTs(N) ≈ FTs(N+1)), both predicates select the same fragment:

• Ts strictly within fragment N (FTs(N) < Ts < LTs(N)): both predicates return fragment N
• Ts at the boundary LTs(N) = FTs(N+1): both predicates return fragment N+1

The bug only manifests when Ts falls in a gap — after LTs(N) but before FTs(N+1). The original predicate returns fragment N (wrong); the new predicate returns fragment N+1 (correct).

In a real environment with continuous message flow, fragments are nearly contiguous and the gap is negligible, which explains why this hasn't been observed in practice. The CI failure exposed it because Timestamp3 was captured during a 200ms sleep, landing in a 406ms gap between two fragments.

The fix is believed to be safe: it does not change behaviour for contiguous fragments, and the property-based tests in unit_SUITE verify correctness for both the contiguous and gap cases across 500 randomly generated manifests.

[the-mikedavis]

Ah yep I think the 🧞 is on the right track here. If a timestamp is between two chunks then we should resolve to the later chunk. (Which is different than offsets, we would resolve to the earlier chunk for a requested offset which is between two chunks.) We should do the same for fragment boundaries 👍

[lukebakken]

Cool I'll keep working on this.

[lukebakken]

AI follow-up: the current code does implement the "prefer later fragment for timestamps" behaviour that @the-mikedavis describes.

With the new predicate Ts >= LTs, partition_point returns the index of the first fragment where Ts >= LTs is false — i.e. the first fragment whose last timestamp exceeds Ts. That is the fragment we want: the earliest fragment that could contain Ts. When Ts falls in a gap between fragments, this correctly resolves to the later fragment, consistent with find_index_position's timestamp behaviour.

[lukebakken]

AI note: two documentation commits were added on top of the main fix.

The first rewrites README.md with a plain-language description of what the plugin does and how it hooks into osiris (replacing the one-liner), fixes the build instructions (wrong directory name), and adds docs/configuration.md documenting all active configuration keys: enabling the plugin, S3 bucket and region, region endpoint overrides, and credential resolution order.

The second fixes four pre-existing issues in docs/README.md: a typo ("are send" → "are sent"), an incorrect unit (fragment max size is 64 MiB not 64 MB), a grammar error ("there prefixes" → "there are prefixes"), and a broken anchor link (#concurrency-control) pointing to a section that was never written.

[the-mikedavis]

Looks great! I have two thoughts

[the-mikedavis]

Looks great, thanks!