fix: notify lighthouse from backfill loop when new subgraph data is found#615
fix: notify lighthouse from backfill loop when new subgraph data is found#615otsybizov wants to merge 1 commit intomainnet-stagingfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR centralizes Lighthouse queue name constants in @chimera-monorepo/utils and updates Cartographer backfill to notify Lighthouse when backfill discovers new subgraph data (deduplicating notifications across operations).
Changes:
- Add
LIGHTHOUSE_QUEUESto utils constants and re-export it (mqclient now re-exports from utils). - Update Cartographer core backfill-related operations to return a
Setof Lighthouse queues to notify, and update handler backfill to aggregate + notify once per queue. - Add a handler-level
mockablere-export layer and introduce unit tests for backfill notification behavior.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/utils/src/constants/queue.ts | Introduces shared LIGHTHOUSE_QUEUES constant in utils. |
| packages/utils/src/constants/index.ts | Re-exports the new queue constants module. |
| packages/adapters/mqclient/src/index.ts | Removes local LIGHTHOUSE_QUEUES definition and re-exports it from utils. |
| packages/agents/lighthouse/src/server.ts | Switches Lighthouse to import LIGHTHOUSE_QUEUES from utils. |
| packages/agents/cartographer/handler/src/notify.ts | Switches Cartographer handler to source LIGHTHOUSE_QUEUES from utils. |
| packages/agents/cartographer/handler/src/processors/tronLogProcessor.ts | Updates import source of LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/solanaInstructionProcessor.ts | Updates import source of LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/monitorProcessor.ts | Removes per-message Lighthouse notifications (settlement/fill) from monitor processor. |
| packages/agents/cartographer/handler/src/processors/invoiceProcessor.ts | Updates import source of LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/intentProcessor.ts | Updates import source of LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/mockable.ts | Adds a re-export shim to allow stubbing core operations in handler tests. |
| packages/agents/cartographer/handler/src/maintenance/backfill.ts | Aggregates queue notifications from backfill operations and notifies Lighthouse once per queue. |
| packages/agents/cartographer/handler/test/maintenance/backfill.spec.ts | Adds unit tests covering backfill operation execution, notification behavior, and error isolation. |
| packages/agents/cartographer/core/src/operations/invoices.ts | Makes hub invoice/deposit operations return queue notification sets for backfill-driven Lighthouse triggers. |
| packages/agents/cartographer/core/src/operations/intents.ts | Makes origin/destination/hub intent operations return queue notification sets for backfill-driven Lighthouse triggers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| it('should deduplicate queue notifications across operations', async () => { | ||
| // Both updateOriginIntents and updateHubIntents return SETTLEMENT | ||
| updateOriginIntentsStub.resolves(new Set([LIGHTHOUSE_QUEUES.INTENT])); | ||
| updateHubIntentsStub.resolves(new Set([LIGHTHOUSE_QUEUES.SETTLEMENT])); | ||
|
|
||
| await runBackfill(context); | ||
|
|
||
| // INTENT should be notified only once | ||
| const intentCalls = notifyStub.getCalls().filter((c: { args: string[] }) => c.args[0] === LIGHTHOUSE_QUEUES.INTENT); | ||
| expect(intentCalls.length).to.equal(1); | ||
| }); |
There was a problem hiding this comment.
The "deduplicate queue notifications" test currently doesn’t actually exercise deduplication: the stubs return two different queues (INTENT and SETTLEMENT), so the test will pass even if notifications are not deduplicated. Update the stubs so at least two operations return the same queue name and assert notifyLighthouse was called only once for that queue (and align the inline comment with the queues being returned).
🤖 Linear
Closes CONG-XXX