Fix duplicate jobs in product creation

[FyreByrd]

Fixes #1373

Summary by CodeRabbit

• Bug Fixes
  • Eliminated duplicate product creation jobs that could occur during workflow initialization through improved job deduplication.
  • Improved workflow initialization with enhanced state snapshot capture to strengthen reliability and recovery capabilities.

[coderabbitai]

Walkthrough

The pull request addresses a product creation stuck state by implementing job deduplication in the BullMQ queue and adding an additional context snapshot during workflow initialization. A unique jobId is injected at enqueue time using the product identifier to prevent duplicate job processing.

Changes

Cohort / File(s)	Summary
Workflow Initialization src/lib/server/workflow/index.ts	Adds a second context snapshot capture immediately after state machine initialization in Workflow.create, prior to database writes
Job Deduplication src/lib/server/workflow/state-machine.ts	Wraps BullMQ.Retry with a jobId parameter composed of JobType.Product_Create and context.productId in the Product_Creation queue enqueue call to prevent duplicate job processing

Sequence Diagram

sequenceDiagram
    participant Workflow as Workflow.create
    participant StateMachine as State Machine
    participant BullMQ as Product_Creation Queue
    
    Workflow->>StateMachine: initiate state machine
    rect rgb(200, 220, 255)
        note over Workflow: NEW: capture snapshot
        Workflow->>Workflow: create context snapshot
    end
    
    Workflow->>BullMQ: enqueue with unique jobId
    rect rgb(200, 220, 255)
        note over BullMQ: NEW: job deduplication
        BullMQ->>BullMQ: check jobId uniqueness
    end
    
    BullMQ-->>Workflow: job queued/deduplicated
    Workflow->>Workflow: proceed with DB writes

Loading

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Areas requiring attention:
  • Verify the snapshot timing and content capture in index.ts doesn't introduce race conditions or unintended state captures
  • Confirm jobId composition (JobType.Product_Create + productId) uniquely identifies duplicate jobs in the intended scope
  • Check BullMQ configuration changes and retry behavior with the new jobId parameter
  • Validate that deduplication doesn't prevent legitimate retry attempts for failed jobs

Suggested Reviewers

• chrisvire

Poem

🐰 A product got stuck, oh what a plight,
With snapshots and deduplication bright,
Each jobId unique, no duplicates stay,
The workflow flows smooth, hip-hip-hooray! 🎉

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Fix duplicate jobs in product creation' directly addresses the main change: adding job deduplication to prevent duplicate product creation jobs, which aligns with the primary objective of fixing the stuck product creation issue.
Linked Issues check	✅ Passed	The PR implements job deduplication in the product creation queue to resolve the stuck product creation process reported in issue #1373, addressing the root cause of duplicate jobs blocking progress.
Out of Scope Changes check	✅ Passed	All changes are scoped to the product creation workflow: job deduplication in state-machine.ts and snapshot capture in workflow/index.ts, with no unrelated modifications detected.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/product-creation/1373

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0b84e1a and 8b887eb.

📒 Files selected for processing (2)

• src/lib/server/workflow/index.ts (1 hunks)
• src/lib/server/workflow/state-machine.ts (1 hunks)

🧰 Additional context used

🧠 Learnings (7)

📓 Common learnings

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1115
File: portal/node-server/BullWorker.ts:74-83
Timestamp: 2025-05-12T13:16:39.942Z
Learning: In BullMQ, using a fixed `jobId` provides idempotency for jobs. Without a `jobId`, each application restart will add a new job with a unique ID to the queue, causing startup jobs to accumulate. However, completed jobs are not automatically reprocessed - they remain in the completed state.

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1115
File: portal/node-server/BullWorker.ts:74-83
Timestamp: 2025-05-12T13:16:39.942Z
Learning: In BullMQ, using a fixed `jobId` provides idempotency for jobs. Without a `jobId`, each application restart will add a new job with a unique ID to the queue, causing startup jobs to accumulate over time. Completed jobs are not automatically reprocessed regardless of whether a `jobId` is specified.

📚 Learning: 2025-09-04T14:26:59.326Z

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1227
File: src/lib/server/job-executors/product.ts:247-250
Timestamp: 2025-09-04T14:26:59.326Z
Learning: In src/lib/server/job-executors/product.ts, the createLocal function's catch block returns false instead of rethrowing errors. This was implemented intentionally to fix another issue, so any changes to this error handling should be carefully evaluated for downstream impacts.

Applied to files:

• src/lib/server/workflow/state-machine.ts

📚 Learning: 2025-09-04T16:23:55.891Z

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1227
File: src/lib/server/job-executors/product.ts:247-250
Timestamp: 2025-09-04T16:23:55.891Z
Learning: In src/lib/server/job-executors/product.ts, createLocal’s catch should log the error via job.log with relevant context (projectId/productDefinitionId/storeId) and still return false to preserve the intentional “no-retry” behavior.

Applied to files:

• src/lib/server/workflow/state-machine.ts

📚 Learning: 2025-05-12T13:16:39.942Z

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1115
File: portal/node-server/BullWorker.ts:74-83
Timestamp: 2025-05-12T13:16:39.942Z
Learning: In BullMQ, using a fixed `jobId` provides idempotency for jobs. Without a `jobId`, each application restart will add a new job with a unique ID to the queue, causing startup jobs to accumulate. However, completed jobs are not automatically reprocessed - they remain in the completed state.

Applied to files:

• src/lib/server/workflow/state-machine.ts

📚 Learning: 2025-05-12T13:16:39.942Z

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1115
File: portal/node-server/BullWorker.ts:74-83
Timestamp: 2025-05-12T13:16:39.942Z
Learning: In BullMQ, using a fixed `jobId` provides idempotency for jobs. Without a `jobId`, each application restart will add a new job with a unique ID to the queue, causing startup jobs to accumulate over time. Completed jobs are not automatically reprocessed regardless of whether a `jobId` is specified.

Applied to files:

• src/lib/server/workflow/state-machine.ts

📚 Learning: 2025-08-04T18:34:58.818Z

Learnt from: 7dev7urandom
Repo: sillsdev/appbuilder-portal PR: 1186
File: src/hooks.server.ts:34-41
Timestamp: 2025-08-04T18:34:58.818Z
Learning: In the AppBuilder Portal codebase, BullMQ workers are only initialized when not building (`if (!building)` in BullWorker constructor), and the graceful shutdown handler in hooks.server.ts also only runs when not building. Therefore, the worker.worker property is guaranteed to be defined in the shutdown context, making optional chaining defensive rather than necessary.

Applied to files:

• src/lib/server/workflow/state-machine.ts

📚 Learning: 2025-09-12T14:57:08.647Z

Learnt from: FyreByrd
Repo: sillsdev/appbuilder-portal PR: 1283
File: src/routes/(authenticated)/tasks/[product_id]/+page.server.ts:252-264
Timestamp: 2025-09-12T14:57:08.647Z
Learning: In src/routes/(authenticated)/tasks/[product_id]/+page.server.ts, FyreByrd prefers to use transition.target[0] prediction mechanism for determining post-send state rather than calling flow.state() after flow.send() due to race condition concerns. The prediction approach is sufficient for redirect decisions between staying on task page vs going to projects page.

Applied to files:

• src/lib/server/workflow/state-machine.ts
• src/lib/server/workflow/index.ts

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: checks / build-and-test

🔇 Additional comments (1)

src/lib/server/workflow/state-machine.ts (1)

308-312: LGTM! Deduplication prevents duplicate job accumulation.

The jobId addition correctly implements BullMQ idempotency to prevent duplicate Product_Create jobs from being enqueued on application restarts or retries. The format ${JobType}_${productId} ensures uniqueness per product while allowing the same product to be processed only once.

Based on learnings