Skip to content

Retry MaxWait 3 seconds is too short! Jitter for 429/TooManyRequests? #1210

Description

@BBirke-Vidispine

MaxWait: 3 * time.Second,

Issue:
Is the MaxWait 3 seconds for retry intended, or was it supposed to be like 3 minutes?
I came across this issue when facing 429/TooManyRequests issues. The code uses the "Retry-After" response header value in this case, which is likely much higher than 3 seconds. Other than in go-retryablehttp, MaxWait is applied outside the Backoff function.
Also, 3 seconds seems not to make much sense for network issues or 5xx errors.

Jitter for 429/TooManyRequests
Shouldn't there also be a jitter added to the "Retry-After" period? Only add, don't execute before "Retry-After" value (other than normal retry jitter)!
Parallel executions with fixed retry period might cause a "thundering herd", all continue at (nearly) the same time.

Effects on existing usage
Execution with retries may take much longer with MaxWait= 3 minutes (up to 15 minutes for 5 retries, but unlikely). The existing 3 seconds default guarantees fast execution, but effectively breaks many retry cases, especially 429/TooManyRequests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions