Skip to content

Conversation

@alkaline-0
Copy link
Contributor

Description

  1. Fix SCD2 sticky boundary behavior: when write_disposition.strategy="scd2" and boundary_timestamp is omitted, always set x-boundary-timestamp to the current load package’s created_at. This ensures each run uses the effective time of that run rather than persisting the previous boundary.
  2. Also updated the SCD2 tests to validate this case

Related Issues

Fixes #3190

rudolfix and others added 23 commits November 18, 2025 23:58
… with seen-null-first=True (#3131)

* child table column removed from parent

* A utility functin that checks whether a column has seen-null-first set

* Improved comments and docstrings, separate method in worker

* null column not inferred if exists as compound

* Column level x-normalizer cleaning moved outside of worker

* Test for empty column becoming compound

* Test clean_seen_null_first_hint
…p using the current load package's creation time when none is provided. Modified corresponding test to ensure correct behavior in the pipeline.
…ests for new behavior, including resetting to current load time.
@alkaline-0 alkaline-0 force-pushed the fix/3190-value-of-boundary-value-remains-unchanged branch from c7797c1 to 80277b1 Compare November 24, 2025 09:53
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Nov 24, 2025

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
docs c087431 Commit Preview URL

Branch Preview URL
Nov 24 2025, 10:00 AM

…chemyMergeFollowupJob to ensure current load package creation time is used when no boundary timestamp is provided. Update DltResourceHints class to streamline timestamp validation for active_record_timestamp and boundary_timestamp. Adjust tests accordingly.
@alkaline-0 alkaline-0 marked this pull request as ready for review November 24, 2025 10:11
@alkaline-0 alkaline-0 requested review from rudolfix and sh-rp November 24, 2025 13:43
@alkaline-0 alkaline-0 closed this Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The value of _dlt_valid_from and _dlt_valid_to keeps being set to a former boundary_value

4 participants