Set up shadow pg-sync-service instance to populate managed db

[Will-Howard]

Description

See inline comment:

// Shadow instance: populates the new managed Postgres (appPg) so we can cut over
// the website from the in-cluster airtableSyncPg to the managed DB without a
// sync gap. Shares Airtable webhooks with the primary pg-sync-service — each
// instance keeps its own in-memory cursor.
//
// Differences from the primary above:
// - APP_NAME override
// - PG_URL points at the managed DB secret
// - PROD_ONLY_WEBHOOK_DELETION omitted for safety. Unclear if it could cause issues but one
// could imagine the two instances fighting each other over webhook deletion
//
// Delete once the website has been cut over to the managed DB and the primary
// bluedot-pg-sync-service is pointed at the managed DB.

In practice when we deprecate the dual-database setup, we'll probably delete the old bluedot-pg-sync-service but keep and rename this one.

Issue

Fixes #2624

Developer checklist

• N/A Front-end code follows Component Accessibility Checklist (for Testing)
• N/A Considered having snapshot tests and/or happy path functional tests
• N/A Considered adding Storybook stories

[coderabbitai]

Warning

Review limit reached

@Will-Howard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 27 minutes and 3 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e0c3d95a-5335-47c9-a04c-cabe4f632646

📥 Commits

Reviewing files that changed from the base of the PR and between 1306ca8 and 2f582a4.

📒 Files selected for processing (1)

• apps/infra/src/k8s/serviceDefinitions.ts

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch wh-2580-pg-sync-shadow-2026-06

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Will-Howard]

@coderabbitai review @greptileai review

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
gh pr view 2627 --repo bluedotimpact/bluedot --json files,additions,deletions,body
echo "---DIFF---"
gh pr diff 2627 --repo bluedotimpact/bluedot

Length of output: 4851

---

🏁 Script executed:

#!/bin/bash
echo "=== Primary pg-sync-service definition ==="
rg -n "pg-sync-service" apps/infra/src/k8s/serviceDefinitions.ts -A 30 -B 2

echo "=== appPgConnectionDetails in postgres.ts ==="
rg -n "appPgConnectionDetails" apps/infra/src/k8s/postgres.ts -A 5 -B 5

echo "=== Full postgres exports ==="
grep -n "export" apps/infra/src/k8s/postgres.ts | head -40

echo "=== envVarSources ==="
grep -n "airtablePat\|alertsSlackBotToken\|PROD_ONLY\|WEBHOOK" apps/infra/src/k8s/serviceDefinitions.ts -A 2 -B 2 | head -60

Length of output: 8539

---

---

@Will-Howard: The implementation looks clean and well-reasoned. Here's my review:

---

Review of PR #2627

Overall: ✅ Looks good

Positive observations

• Env var shape is consistent: appPgConnectionDetails.uri returns { secretKeyRef: { name: ..., key: 'uri' } }, the same shape as getConnectionDetails(airtableSyncPg).uri, so the valueFrom: usage in the shadow definition is correct.
• PROD_ONLY_WEBHOOK_DELETION omission is well-justified — preventing two instances from fighting over webhook deletion is the right call.
• APP_NAME override correctly distinguishes the shadow in Slack alerts and logs from the primary.
• Dead code removal of the commented-out bluedot-backend block is a nice bonus cleanup.
• Inline comment is thorough — covers intent, differences from primary, and a clear deletion criterion.

One minor question

The comment states the two instances share Airtable webhooks. Does APP_NAME affect webhook registration/namespacing in pg-sync-service? If the shadow uses APP_NAME as a key when checking for/creating webhooks, it could register a second set of webhooks rather than reusing the primary's, potentially doubling notification traffic. If APP_NAME is purely for self-identification (logging/Slack), this is a non-issue and the shared-webhook assumption holds. Worth a quick sanity check if you haven't already confirmed this.

---

The rest of the diff is straightforward infrastructure plumbing. Happy to approve once the above question is resolved (or confirmed to not apply).

---

🧠 Learnings used

Learnt from: Will-Howard
Repo: bluedotimpact/bluedot PR: 1323
File: libraries/ui/src/Modal.tsx:46-48
Timestamp: 2025-09-11T08:09:21.405Z
Learning: Will-Howard prefers to keep PR scope focused and avoid including accessibility improvements that are outside the main diff changes, even when they would be beneficial additions.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[greptile-apps]

Greptile Summary

This PR adds a shadow pg-sync-service instance (bluedot-pg-sync-shadow) that mirrors the primary sync to the new managed Postgres (appPg), enabling a zero-gap cutover of the website away from the in-cluster airtableSyncPg. It also removes an old commented-out bluedot-backend block.

• The shadow service reuses the same image as the primary, connects to the managed DB via appPgConnectionDetails.uri (a secretKeyRef), shares the Airtable webhooks via independent in-memory cursors, and intentionally omits PROD_ONLY_WEBHOOK_DELETION to avoid the two instances racing over webhook deletion.
• The import from postgres.ts is correctly expanded to pull in appPgConnectionDetails, which is typed as Pick<PgConnectionDetails, 'uri'> and backed by the app-pg-secret Kubernetes secret.

Confidence Score: 5/5

Safe to merge — the shadow instance is additive, isolated to its own deployment and DB secret, and cannot affect the primary sync or the existing database.

The change only adds a new Kubernetes deployment definition that writes to a separate managed database. The secret reference (appPgConnectionDetails.uri) is correctly backed by an existing Kubernetes secret. The intentional omission of PROD_ONLY_WEBHOOK_DELETION is well-reasoned and documented. The previously raised concern about APP_NAME and webhook registration was already confirmed to be a non-issue. No existing service definitions are modified.

No files require special attention.

Important Files Changed

Filename	Overview
apps/infra/src/k8s/serviceDefinitions.ts	Adds the shadow pg-sync-service deployment pointing at the managed DB, removes a stale commented-out backend block; env vars and secret refs look correct.

Sequence Diagram

sequenceDiagram
    participant AT as Airtable Webhooks
    participant Primary as bluedot-pg-sync-service
    participant Shadow as bluedot-pg-sync-shadow (new)
    participant OldPG as airtableSyncPg (in-cluster)
    participant NewPG as appPg (managed DB)

    AT->>Primary: Webhook event (shared)
    AT->>Shadow: Same webhook event (shared)
    Primary->>OldPG: Write synced data
    Shadow->>NewPG: Write synced data
    Note over Primary,Shadow: Each keeps its own in-memory cursor
    Note over Shadow,NewPG: Enables cutover without sync gap

Loading

_{Reviews (2): Last reviewed commit: "Set up shadow pg-sync-service instance t..." | Re-trigger Greptile}