Skip to content

Update tables.md #7652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from
Open

Update tables.md #7652

wants to merge 7 commits into from

Conversation

ksaha873
Copy link
Contributor

@ksaha873 ksaha873 commented May 30, 2025

Proposed changes

Adding note on Timestamp column being null in Profiles tables in case of backfill. This is expected behaviour and happens because we infer historical profile changes from the current state of the profile.

More context: Slack thread.

CSE requested change in documentation here: JIRA.

Merge timing

ASAP once approved.

Related issues (optional)

JIRA ticket: link.

@ksaha873 ksaha873 requested review from pwseg and a team as code owners May 30, 2025 12:01
Raw tables contain complete historical data when using historical backfill.
Raw tables contain complete historical data when using historical backfill.

Note: `Timestamp` column will be empty for backfilled data. This is because we reverse engineer historical profile changes from the current state of the profile, and do not refer to the actual change history.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we do not want to set to the current timestamp also? Also what is this timestamp? What does it refer to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp refers to the instant at which the event wad ingested into Segment. For backfill, we don't refer to individual events, but to the final profile state. So we don't have access to the original event timestamps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants