Imagine an upstream API starts returning 503s, and every client computes the same retry schedule. The moment it starts coming back, all of them retry at once and knock it over again before it finishes recovering.
Exponential backoff is the standard defense, though it only spreads those retries across a wider window. The total number of requests hitting the upstream stays the same.
This guide covers the formula, where jitter helps, the control that bounds retry volume, and which Vercel primitives retry on your behalf.
Key takeaways:
Backoff spaces retries further apart, but the same number of requests still arrive at the failing service, so the fix for a retry storm is a limit on how much of your traffic can be retried.
Clients that fail at the same moment wait the same amount of time and retry together, so each delay needs a random component to break up the pattern.
Retrying an expired API key or a malformed request wastes every attempt, so check whether waiting can resolve the failure before scheduling a retry.
Retry sits at a different layer for each Vercel primitive, so check whether the one you're using already retries before adding your own loop.
Circuit breakers need more failure data than a short-lived function instance ever collects, which is why AI Gateway makes those calls using data from across the fleet.
Copy link to headingWhat is exponential backoff?
Exponential backoff is a retry strategy that waits longer after each failed attempt, usually doubling the delay every time. You reach for it when a request fails for a reason that might clear on its own, such as a 429 rate limit, a 503 service unavailable, or a network timeout, because retrying immediately piles more load onto a service that is already struggling.
Each longer wait gives the upstream more room to recover, and the strategy stops being useful the moment a failure becomes permanent, since no amount of waiting can fix a 401 unauthorized or a 400 bad request.
Copy link to headingHow exponential backoff works with jitter
The formula alone has a defect that appears only under correlated failure. Every client that failed at the same instant computes an identical schedule, so they retry together at 1 second, then together at 2 seconds, and the wave gets wider rather than smaller.
Randomizing each delay, called jitter, is what breaks that synchronization. Google Cloud's June 2025 outage shows the cost. Service Control tasks restarted without randomized exponential backoff and created a herd effect on the Spanner table underneath, and recovery in us-central1 took around 2 hours and 40 minutes against roughly 40 minutes elsewhere, because engineers had to throttle task creation to keep the underlying infrastructure alive.
Four jitter variants dominate production use:
Decorrelated Jitter has a failure mode worth knowing about. Once prev * 3 consistently exceeds the cap, the output pins to the cap and stops randomizing, which returns clients to the synchronized schedule the jitter existed to prevent.
Jitter only changes when each retry arrives. The same number of requests still reaches the upstream, and on serverless, each of those requests runs inside a function with a duration limit of its own.
Copy link to headingBenefits of exponential backoff for serverless teams
Backoff decides whether a struggling upstream recovers. On serverless, it also decides whether your own function survives the schedule it is running.
A growing delay affects five things at the same time:
Recovery time for the upstream: Each doubled wait gives the failing service a longer uninterrupted stretch to drain queues, restart tasks, or add capacity. That recovery is what ends the incident, not the retries arriving during it.
Protection against a synchronized wave: Randomized delays scatter clients that all failed at the same moment across a window of time. Without that, a service comes back up and gets knocked down again by everyone who was waiting for it.
Room inside the function's time limit: A function on Fluid compute runs for 300 seconds by default, and up to 800 seconds on Pro and Enterprise. A retry schedule whose total wait passes that limit ends the invocation with
FUNCTION_INVOCATION_TIMEOUT.Cost of the time spent waiting: Active CPU billing runs while your code executes and pauses while the function waits on I/O, such as model calls, database queries, and third-party APIs. A long wait on an upstream response costs less than the wall-clock number suggests.
Safety of running the work twice: A retry is only safe when the same operation can run again without changing the result. GET, PUT, HEAD, DELETE, and OPTIONS are idempotent methods, while POST and PATCH need an idempotency key before they are safe to repeat.
Function duration places a separate bound on the retry schedule. Waits of 1, 2, 4, 8, 16, and 30 seconds consume 61 seconds of sleep before the seventh attempt, which fits comfortably under a 300-second default and not at all under a maxDuration that someone dropped to 10 seconds for cost reasons.
Once retries need minutes rather than seconds, the function is the wrong place for them, and the retry loop becomes a set of components chosen deliberately.
Copy link to headingCore components of an exponential backoff retry policy
A retry policy is five decisions, and most incident reports trace back to one of them being left at a default. Each is worth setting explicitly rather than inheriting.
Copy link to headingBase delay, multiplier, and cap
Three numbers define the schedule. The base delay is how long you wait after the first failure, the multiplier is what that wait gets multiplied by each time after, and the cap is the longest wait you allow. Vercel's Workflow error handling course uses Math.min(1000 * 2^(attempt-1), 30000), which is a 1-second base, a multiplier of two, and a 30-second cap, producing waits of 1, 2, 4, 8, and 16 seconds.
The cap is what keeps the schedule usable. Without it, doubling reaches waits no one will sit through and no function will survive, so every attempt past the cap waits that maximum instead. The base is the number worth setting deliberately.
A database that fails over in under a second and a model provider shedding load for a minute need different starting waits, so you set it against the upstream's real recovery time rather than copying 1,000 milliseconds out of habit.
Copy link to headingJitter
Jitter is randomization applied to the computed delay so that clients failing at the same moment do not retry at the same moment. It costs one line of code and is the highest-value addition to a naive backoff loop, and it belongs at every layer that computes a delay, including reconnect loops.
Vercel's WebSocket documentation shows the reconnect pattern as reconnectDelay = Math.min(reconnectDelay * 2, 30000), which doubles correctly and does not randomize.
A WebSocket served by a Vercel Function closes when the function hits its maximum duration, so connected clients get disconnected on roughly the same cadence, which is precisely the condition that produces a synchronized reconnect wave. The cost of adding randomness is that individual completion times become unpredictable, and Full Jitter can produce a near-zero wait on any given attempt.
Copy link to headingError classification
Error classification decides whether a failure consumes a retry attempt at all. Retrying a 401 is wasted compute in the best case and an account lockout in the worst.
The split follows one question, which is whether waiting can change the outcome:
Vercel Workflows makes this decision explicit through two error classes, and a hand-rolled loop should default to failing rather than retrying, so unclassified errors surface instead of quietly burning attempts.
Copy link to headingAttempt limits
An attempt limit caps how many times a single logical request can be retried. Google's Site Reliability Engineering (SRE) practice sets a per-request budget of three attempts, on the reasoning that a request that has already landed on overloaded tasks three times is unlikely to find a healthy one on the fourth.
Ecosystem defaults cluster in the same range. The AI SDK, Vercel's software development kit (SDK) for building AI applications, ships maxRetries: 2, for three total attempts, and Vercel Workflows retries a failed step three times by default, for four total attempts.
This is also the limit that matters least, because three attempts per request bounds one user's experience and does nothing about aggregate load. A hundred thousand clients each stopping politely at three attempts still produce three hundred thousand requests against a service that cannot serve one.
Copy link to headingRetry budget
A retry budget caps the fraction of outgoing requests allowed to be retried. Each client tracks the ratio of retries to total requests across a recent window and refuses to retry once that ratio crosses the threshold, failing locally without touching the network.
The growth numbers are why this one is worth building. In Google's worst case, a datacenter rejecting a large portion of requests sees traffic grow to almost three times the original rate under a per-request limit of three attempts, and layering a 10% retry budget on top reduces that growth to roughly 1.1x.
A budget is a one-time addition to application code, and it is the only component here that bounds retry volume rather than retry timing. Everything else on this list makes the same number of retries arrive more politely.
Copy link to headingHow Vercel handles exponential backoff across its retry primitives
Retry behavior on Vercel is set per primitive, so implementing a retry loop at a layer that already retries is the most common way teams create compounding by accident. This map is worth checking before writing any retry code:
The row that catches teams out is cron jobs, where a failed invocation is lost until the next scheduled run and nothing in the platform brings it back.
Copy link to headingVercel Workflows makes the retryable-or-fatal decision explicit
A hand-rolled retry loop buries its classification logic inside a catch block, where it drifts out of sync with the upstream's real error semantics and nobody notices until an auth failure quietly consumes four attempts. Vercel Workflows moves that decision into the type system, where throwing a FatalError skips retry logic entirely and throwing a RetryableError schedules another attempt after a delay you specify.
A step that classifies before backing off looks like this:
import { FatalError, RetryableError, getStepMetadata } from 'workflow';
export async function callUpstream(payload: Payload) { 'use step';
const response = await fetch(endpoint, { method: 'PUT', body: payload });
if (response.status === 401 || response.status === 400) { throw new FatalError(`Not retryable: ${response.status}`); }
if (!response.ok) { const { attempt } = getStepMetadata(); throw new RetryableError(`Upstream returned ${response.status}`, { retryAfter: Math.min(1000 * 2 ** (attempt - 1), 30000), }); }
return response.json();}The retryAfter option accepts a duration string such as "5m", a millisecond count, or a Date, so an upstream's own rate-limit signal can drive the wait instead of a formula. getStepMetadata() exposes the attempt number, which makes attempt-aware backoff possible without tracking state yourself. Workflows can also pause, resume, and hold state for minutes to months without a run-level duration limit, so a retry schedule that needs to outlive one function invocation has somewhere to live.
Copy link to headingVercel Queues forces backoff and expects idempotent consumers
Queue consumers get written against the happy path, and the first duplicate delivery corrupts something that was never designed to run twice. Vercel Queues delivers at least once, with redelivery following ordinary causes such as a consumer crash, a deployment rollout, or a function timing out before it acknowledges. Queues respect your configured retry delay for the first 32 delivery attempts, then force exponential backoff to prevent runaway deliveries.
There is no built-in dead-letter queue, so poisoned messages get handled at the application level through the SDK's retry callback, and because messages with no prior delivery attempts are always prioritized over retried ones, a failing message falls to lower priority on its own instead of blocking the consumer. Underneath all of this, your handler has to survive running twice, which runs deeper than backoff and is covered in our API idempotency guide. The message lifetime is the other constraint to plan around, where the default TTL is 24 hours, and the maximum is 7 days.
Copy link to headingAI Gateway moves retry and fallback out of application code
Multi-provider AI applications accumulate retry logic, fallback routing, and provider health checks in application code, and every one of those is an estimation problem that individual function instances are badly positioned to solve. AI Gateway handles all three at the routing layer, retrying the same model on a different provider when one degrades, with no fallback array configured in advance.
Zo Computer ran both paths simultaneously during rollout, producing a live A/B comparison under identical production conditions. In the after-switch period, the non-Vercel route recorded a 17.07% retry rate and a 10.38% POST error rate, compared with 0.34% and 0.45%, respectively, on Vercel. The average number of attempts per Vercel-routed chat was 1.00. The case study’s 20x improvement compares the 0.34% Vercel retry rate with Zo’s 7.52% pre-switch baseline.
Cline, the open-source coding agent, saw the same shape when it A/B tested its previous router against AI Gateway on live production traffic, with P99 streaming latency improving 10 to 14% and API error rates dropping 43.8%.
Copy link to headingFluid compute sets the wall-clock budget every retry schedule runs inside
A retry schedule gets designed against an upstream's recovery time and then deployed into a function whose ceiling was configured for unrelated reasons. Vercel Functions have no built-in retry, so the whole schedule runs inside one invocation. The Fluid compute default is 300 seconds on all plans, with a maximum of 800 seconds on Pro and Enterprise. Supported Node.js and Python runtimes can opt into an extended 1,800-second maximum in beta. Active CPU billing pauses while the function waits on documented I/O, which lowers the cost of waiting without moving the wall-clock ceiling.
When a retry schedule needs the headroom, configure maximum duration on the specific function rather than as a project default:
{ "$schema": "https://openapi.vercel.sh/vercel.json", "functions": { "app/api/sync/**/*": { "maxDuration": 800 } }}Anything that needs to keep retrying past that ceiling belongs in Vercel Workflows rather than in a longer function.
Copy link to headingFive best practices for implementing exponential backoff and jitter
Those five components are settings. Turning them into a policy that holds during a real incident takes a handful of decisions about where the retry lives and what stops it.
Copy link to headingClassify errors before you calculate a delay
Loops that treat every exception alike spend their full attempt budget on an expired API key and then report a timeout. The failure reads as a slow upstream in the logs, and finding the real cause costs an hour.
You want that check running ahead of the sleep rather than inside the catch block after it. Retryable failures get the backoff schedule, fatal failures throw immediately, and anything you have not classified fails fast so it reaches you as an error rather than as latency.
Copy link to headingHonor upstream retry signals over your own formula
The doubling schedule is a guess about when capacity returns. Some upstreams say so directly, and overriding that signal either retries too early and burns an attempt or waits far longer than needed.
Vercel's own 429 responses report the rate the caller is ramping toward, which means a correctly timed first retry succeeds. The @vercel/blob SDK follows the same pattern, where a BlobServiceRateLimited error carrying error.retryAfter replaces the computed delay outright.
Copy link to headingRetry at the layer immediately above the failure, and nowhere else
Retries implemented at every layer compound multiplicatively rather than additively. Four attempts at each of three layers produce 4³, or 64 requests reaching the database for a single user action, which converts a small error rate into an outage.
Google's SRE guidance is to retry only at the layer immediately above the one rejecting the request, and to pass an explicit do-not-retry error upward once a request is exhausted. Before you tune any of those layers, work out which of them already retry, because that audit usually finds more than the tuning does.
Copy link to headingCap the retry fraction, not only the retry count
Retry volume has a threshold past which a system enters a metastable failure state that it cannot leave on its own. The numbers make it concrete. An application running at 280 queries per second against a database that handles 300 hits a 10-second network interruption, the offered load doubles to 560 on recovery, goodput falls to zero, and it stays there after the interruption clears. Getting out requires dropping the load below 150 queries per second.
No backoff schedule moves any of those numbers, because the offered load is identical; it gets spaced out. A budget moves them by refusing to send, so some of your requests fail locally and never reach the network. Accepting those local failures is the point rather than a side effect.
Copy link to headingPrefer a retry budget to a circuit breaker on short-lived instances
A circuit breaker works by accumulating enough failure observations inside one client instance to estimate the true failure rate. Vercel distributes traffic across short-lived function instances that each see low individual volume, so per-instance estimates are noisy and the breaker either trips early or never trips.
Google's SRE book flags the same constraint for client-side throttling, noting that clients sending requests only sporadically have a drastically reduced view of backend state. A ratio-based budget degrades more gracefully under that constraint, and moving the decision into a routing layer removes the estimation problem entirely.
Copy link to headingShip retry logic that survives a bad upstream day on Vercel
A maxRetries: 2 default bounds a single call. Whether a struggling service recovers is decided elsewhere, by the fraction of traffic allowed to be retried, the classification that keeps permanent failures from consuming attempts, and the layer that owns the retry in the first place. How much of that you write yourself depends on which primitives you build on.
Here's how Vercel handles retry behavior across the primitives engineering teams already run:
Vercel Workflows: The type system enforces error classification rather than convention, so a permanent failure fails fast and a retryable one waits as long as the upstream asked it to.
AI Gateway: Provider failover happens on failure of data from the whole fleet, which signals that no single function instance is positioned to collect.
Vercel Queues: Delivery is at least once with backoff enforced by the platform, so a consumer that crashes mid-message gets the work again instead of dropping it.
Fluid compute on Vercel Functions: Waiting on an upstream stops billing Active CPU, which changes what a long retry schedule costs to run.
ISR: A failed revalidation keeps serving stale content instead of an error, so a broken upstream never becomes a broken page.
Start a new project to put these retry patterns into practice, or browse Vercel templates for examples already wired up with Workflows and AI Gateway.
Copy link to headingFrequently asked questions about exponential backoff
Copy link to headingWhat is the formula for exponential backoff?
The canonical formula is Math.min(1000 * 2**(attempt-1), 30_000), which produces waits of 1, 2, 4, 8, and 16 seconds and then caps at 30 seconds. The cap prevents unbounded doubling from producing waits; no function invocation can survive.
Copy link to headingWhat is the difference between Full Jitter and Decorrelated Jitter?
Full Jitter picks a random delay between zero and the capped exponential value, producing the lowest client work of the four common variants. Decorrelated Jitter completes faster but issues more calls, and it stops randomizing once its computed interval repeatedly exceeds the cap.
Copy link to headingDo Vercel Functions retry failed requests automatically?
No. Vercel Functions and cron jobs have no automatic retry, so the application code owns the logic. The schedule runs inside the function's configured wall-clock duration, which defaults to 300 seconds and maxes out at 800 seconds on Pro and Enterprise.
Copy link to headingHow many times does Vercel Workflows retry a failed step?
Vercel Workflows retries a failed step 3 times by default, for a total of 4 attempts. Throwing a FatalError skips retries entirely, while a RetryableError retries after the delay has passed through its retryAfter option.
Copy link to headingDoes the AI SDK retry failed model calls by default?
Yes. maxRetries: 2 is the default on core functions, including generateText, streamText, and embedMany, for three total attempts. Set maxRetries: 0 to disable retries, which is useful when a gateway or workflow layer already handles them.