Skip to content

feat(revenue_recovery): Add redis-based payment processor token tracking for revenue recovery #8846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 81 commits into
base: main
Choose a base branch
from

Conversation

aniketburman014
Copy link
Contributor

@aniketburman014 aniketburman014 commented Aug 5, 2025

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

This module implements a Redis-backed system for managing and selecting the best payment processor token during payment retries.

It ensures:

  • Safe token usage (no race conditions with locks) locked on connector customer level - if a payment has been scheduled then all other invoices for that customer will be in a queue.
  • Consistent retry limit enforcement

Why This Exists

In the revenue recovery flow, there is no direct mapping between a customer and their payment methods.

However, card networks enforce retry limits (daily and rolling 30-day) for each merchant–customer card.
Without tracking at this level, retries could breach these limits, leading to:

  • Higher decline rates
  • Potential network penalties

The Solution

  1. Group multiple cards of a customer under a single connector_customer_id.
  2. Store all payment processor tokens for that connector customer in Redis.
  3. Allow tokens to be:
    • Inserted when a webhook is received
    • Retrieved during the process tracker’s calculation flow to select the best retry candidate

This enables:

  • Enforcement of daily and 30-day rolling thresholds
  • Compliance with network-level retry rules
  • Token availability across flows without additional database lookups

Redis Schema

Key Pattern Type Purpose
customer:{customer_id}:status String Lock key storing the payment_id to prevent parallel updates
customer:{customer_id}:tokens Hash Maps token_idTokenStatus JSON

Example TokenStatus JSON

{
  "token_id": "tok_abc123",
  "error_code": "do_not_honor",
  "network": "visa",
  "added_by_payment_id": "pay_xyz789",
  "retry_history": {
    "2025-08-01": 1,
    "2025-08-02": 2
  }
}

Flow

Step 1 – Acquire Lock

  • Create a lock key:
    customer:{customer_id}:status → value = payment_id
  • Purpose: Prevent parallel processes from working on the same customer
  • If the lock is already present:
    • Exit early with message: "Customer is already locked by another process."

Step 2– Fetch Existing Tokens

  • Retrieve all payment processor tokens from
    customer:{customer_id}:tokens
  • These tokens are then evaluated for retry eligibility

Step 3 – Check Retry Limits

For each token:

  • Count retries in the last 30 days
  • Check if daily or rolling 30-day limit is exceeded
  • If limit exceeded → give a wait time for those tokens
  • Calculate wait time before the token can be retried again

Step 4 – Decider Logic (Upcoming)

  • For all eligible tokens:
    1. Determine the ** schedule time** for the psp tokens
    2. Select the token with the earliest schedule time

Step 5 – Update Retry State

  • After Transaction Executed
  • Increment today’s retry count for the selected token
  • Persist the updated TokenStatus back to Redis

Step 6 – Release Lock

  • Delete customer:{customer_id}:status lock key
  • Allow the next payment flow to proceed

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Checklist

  • I formatted the code cargo +nightly fmt --all
  • addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@aniketburman014 aniketburman014 requested a review from a team as a code owner August 5, 2025 21:28
Copy link

semanticdiff-com bot commented Aug 5, 2025

Review changes with  SemanticDiff

Changed Files
File Status
  api-reference/v1/openapi_spec_v1.json  67% smaller
  api-reference/v2/openapi_spec_v2.json  67% smaller
  crates/router/src/core/payments/transformers.rs  51% smaller
  crates/common_enums/src/enums.rs  23% smaller
  crates/router/src/core/revenue_recovery/transformers.rs  19% smaller
  crates/router/src/workflows/revenue_recovery.rs  11% smaller
  crates/router/src/core/webhooks/recovery_incoming.rs  6% smaller
  crates/router/src/types/storage/revenue_recovery.rs  6% smaller
  crates/hyperswitch_connectors/src/connectors/stripebilling/transformers.rs  4% smaller
  config/deployments/integration_test.toml Unsupported file format
  config/deployments/production.toml Unsupported file format
  config/deployments/sandbox.toml Unsupported file format
  config/development.toml Unsupported file format
  crates/hyperswitch_connectors/src/connectors/chargebee/transformers.rs  0% smaller
  crates/hyperswitch_connectors/src/connectors/recurly/transformers.rs  0% smaller
  crates/hyperswitch_domain_models/src/payments.rs  0% smaller
  crates/hyperswitch_domain_models/src/revenue_recovery.rs  0% smaller
  crates/hyperswitch_domain_models/src/router_response_types/revenue_recovery.rs  0% smaller
  crates/router/src/core/errors.rs  0% smaller
  crates/router/src/core/revenue_recovery/api.rs  0% smaller
  crates/router/src/types/storage.rs  0% smaller
  crates/router/src/types/storage/revenue_recovery_redis_operation.rs  0% smaller
  loadtest/config/development.toml Unsupported file format
  proto/recovery_decider.proto Unsupported file format

@aniketburman014 aniketburman014 requested a review from Copilot August 5, 2025 21:29
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements Redis-based token management functionality for revenue recovery with network-specific retry limits. The changes add comprehensive Redis operations for managing payment processor tokens with customer locking mechanisms and configurable retry thresholds per card network.

  • Adds Redis-based token management with customer locking and retry tracking
  • Implements network-specific retry limits configuration for different card types
  • Changes default feature flag from "v1" to "v2" in Cargo.toml

Reviewed Changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
crates/router/src/types/storage/revenue_recovery_redis_operation.rs New module implementing comprehensive Redis token management with async operations, customer locking, and retry logic
crates/router/src/types/storage/revenue_recovery.rs Adds network-specific retry configuration structures and network type enum
crates/router/src/workflows/revenue_recovery.rs Imports new Redis token manager and adds commented placeholder function
crates/router/src/types/storage.rs Registers new revenue_recovery_redis_operation module
crates/router/Cargo.toml Changes default feature from "v1" to "v2"
Config files Adds card network retry limits configuration across all environments

@aniketburman014 aniketburman014 marked this pull request as draft August 6, 2025 06:35
@aniketburman014 aniketburman014 self-assigned this Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@juspay juspay deleted a comment from Copilot AI Aug 6, 2025
@aniketburman014 aniketburman014 marked this pull request as ready for review August 6, 2025 17:18
@aniketburman014 aniketburman014 changed the title Redis manipulation feat(revenue_recovery): Added logic for adding payment method token to redis for status and retry mapping Aug 6, 2025
@aniketburman014 aniketburman014 requested review from a team as code owners August 6, 2025 21:08
connector_customer_id,
)
.await
.map_err(|_| errors::ProcessTrackerError::EApiErrorResponse)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why discarding error? change_context to EApiErrorResponse

}
.change_context(errors::ProcessTrackerError::EApiErrorResponse)?;

let attempts_response = Box::pin(payments::payments_list_attempts_using_payment_intent_id::<
Copy link
Member

@jarnura jarnura Aug 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not use this api to get the list of response, we should use normal get with the expanding the attempts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would need this api, as we won't have active_attempt_id. Active attempt id is mandatory for get payments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For call_psync_api only payment_id is required

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a validation in get payment api, which require active_payment_attempt_id as mandatory field.
but in our flow we only update active_payment_attempt_id when reaching terminal state.
Screenshot 2025-08-12 at 2 15 31 PM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow the way how call_psync_api is implemented, @Aprabhat19 said there is equivalent api for the below implementation also available

payment_id: &id_type::GlobalPaymentId,
connector_customer_id: &str,
) -> CustomResult<Option<ScheduledToken>, errors::ProcessTrackerError> {
let payment_intent_response = Box::pin(payments::payments_intent_core::<
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These api related things are already defined in revenue_recovery api's reuse them

}
.change_context(errors::ProcessTrackerError::EApiErrorResponse)?;

let attempts_response = Box::pin(payments::payments_list_attempts_using_payment_intent_id::<
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow the way how call_psync_api is implemented, @Aprabhat19 said there is equivalent api for the below implementation also available

jarnura
jarnura previously approved these changes Aug 13, 2025
Aniket Burman and others added 2 commits August 13, 2025 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
M-api-contract-changes Metadata: This PR involves API contract changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Retry Limit Tracking for Payment Processor Tokens in Revenue Recovery
4 participants