fix: notify lighthouse from backfill loop when new subgraph data is found#616
fix: notify lighthouse from backfill loop when new subgraph data is found#616otsybizov wants to merge 1 commit intomainnet-stagingfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR centralizes Lighthouse queue name constants and updates Cartographer’s backfill maintenance loop to notify Lighthouse when backfill operations discover new subgraph data, with accompanying refactors/tests to support this flow.
Changes:
- Moved
LIGHTHOUSE_QUEUESto@chimera-monorepo/utils(and re-exported frommqclient) to avoid duplication and allow core ops to reference queue names. - Updated Cartographer core operations to return a
Setof Lighthouse queues to notify when they pull new data. - Updated Cartographer handler backfill maintenance to aggregate/deduplicate queue notifications and enqueue Lighthouse jobs accordingly, with new unit tests.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/utils/src/constants/queue.ts | Introduces centralized LIGHTHOUSE_QUEUES constants in utils. |
| packages/utils/src/constants/index.ts | Exposes the new queue constants via the utils constants barrel. |
| packages/agents/lighthouse/src/server.ts | Switches Lighthouse to import LIGHTHOUSE_QUEUES from utils. |
| packages/agents/cartographer/handler/test/maintenance/backfill.spec.ts | Adds unit tests for backfill-triggered Lighthouse notifications/dedup/failure behavior. |
| packages/agents/cartographer/handler/src/processors/tronLogProcessor.ts | Updates import source for LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/solanaInstructionProcessor.ts | Updates import source for LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/monitorProcessor.ts | Removes direct Lighthouse notifications for settlement/fill message processing. |
| packages/agents/cartographer/handler/src/processors/invoiceProcessor.ts | Updates import source for LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/processors/intentProcessor.ts | Updates import source for LIGHTHOUSE_QUEUES. |
| packages/agents/cartographer/handler/src/notify.ts | Uses utils as the source of LIGHTHOUSE_QUEUES for queue initialization. |
| packages/agents/cartographer/handler/src/mockable.ts | Adds a mockable re-export layer for core ops to enable stubbing in handler tests. |
| packages/agents/cartographer/handler/src/maintenance/backfill.ts | Aggregates queue names returned by notify-capable ops and notifies Lighthouse once per queue. |
| packages/agents/cartographer/core/src/operations/invoices.ts | Returns Set<string> indicating INVOICE queue notification when invoice/deposit data is found. |
| packages/agents/cartographer/core/src/operations/intents.ts | Returns Set<string> indicating INTENT/FILL/SETTLEMENT notifications when new intent data is found. |
| packages/adapters/mqclient/src/index.ts | Removes local queue constants and re-exports LIGHTHOUSE_QUEUES from utils for compatibility. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| afterEach(() => { | ||
| restore(); | ||
| }); |
There was a problem hiding this comment.
This file’s afterEach calls sinon.restore(), but the cartographer-handler test runner already loads test/globalTestHook.ts, which runs restore() (and reset()) after every test. Consider removing this local afterEach to avoid redundant restores and keep cleanup consistent across the suite.
| let updateSettlementIntentsStub: SinonStub; | ||
| let updateHubInvoicesStub: SinonStub; | ||
| let updateHubDepositsStub: SinonStub; | ||
| let updateMessagesStub: SinonStub; | ||
|
|
||
| // Core operations that should NOT trigger lighthouse notifications |
There was a problem hiding this comment.
The test comment/grouping is misleading: updateSettlementIntentsStub is listed under “operations that should trigger lighthouse notifications”, but updateSettlementIntents is part of the non-notifying plainOperations in runBackfill. Update the comment (or move the stub declaration) so the test intent matches the implementation.
| let updateSettlementIntentsStub: SinonStub; | |
| let updateHubInvoicesStub: SinonStub; | |
| let updateHubDepositsStub: SinonStub; | |
| let updateMessagesStub: SinonStub; | |
| // Core operations that should NOT trigger lighthouse notifications | |
| let updateHubInvoicesStub: SinonStub; | |
| let updateHubDepositsStub: SinonStub; | |
| // Core operations that should NOT trigger lighthouse notifications | |
| let updateSettlementIntentsStub: SinonStub; | |
| let updateMessagesStub: SinonStub; |
🤖 Linear
Closes CONG-XXX