Retries Done Right: Backoff, Jitter, and the Thundering Herd

A naive retry loop can turn a small outage into a full one. Learn how exponential backoff and jitter keep a failing system from getting worse.

Every developer reaches for the same reflex when a network call fails: try again. It usually works, the request succeeds on the second attempt, and you move on. But that reflex hides a sharp edge. When a dependency is already struggling, a naive retry loop doesn't help it recover — it piles on, turning a brief hiccup into a sustained outage. Getting retries right is one of the cheapest reliability wins available, and most teams leave it on the table.

The retry storm

Imagine a payment API slows down for two seconds. A thousand clients are mid-request. Their calls time out, and every one of them immediately retries. Now the struggling API receives its normal load plus a thousand instant retries — and a moment later, when those time out too, another thousand. This is the thundering herd: the exact moment a service is weakest is the moment its callers hit it hardest.

The cruel part is that immediate retries feel responsible. You handled the error! But a fixed, instant retry is often worse than no retry at all, because it converts transient slowness into a self-sustaining overload that outlives the original problem.

Back off exponentially

The first fix is to wait longer between attempts, and to grow that wait on each failure: 1 second, then 2, then 4, then 8. This is exponential backoff. It hands the downstream system breathing room that widens automatically the longer the trouble lasts.

A simple shape:

delay = base x 2^(attempt number), capped at a maximum

The cap matters. Without it, the eighth retry might wait several minutes, which is rarely what a user wants. A typical setup caps the delay at something like 30 or 60 seconds and limits the total number of attempts to a handful.

Add jitter

Backoff alone has a subtle flaw. If all thousand clients failed at the same instant, they will also back off by the same amount — and retry at the same instant again. You've spread the herd out in time but kept it perfectly synchronized. The spikes are smaller, but they're still spikes.

The fix is jitter: add randomness to each delay so clients scatter instead of marching in lockstep. Rather than waiting exactly 4 seconds, each client waits a random duration up to 4 seconds. It's a tiny change — a single random call — and it's the difference between smooth, even pressure on a recovering service and a series of synchronized hammer blows.

Guardrails that matter

A few rules keep retries safe in production:

• Only retry what's safe to repeat. Reads are usually fine. Writes need idempotency keys so a retried request can't charge a customer twice.
• Only retry transient failures. A 500 or a timeout is worth retrying; a 400 or a 401 will fail identically every time, so don't waste attempts on it.
• Respect Retry-After. When a server tells you how long to wait, honor it instead of guessing.
• Set a retry budget. Cap retries as a fraction of total traffic so a widespread failure can't multiply your own load.

The takeaway

Retries are not free, and "just try again" is not a strategy. Exponential backoff gives a failing system room to recover, jitter keeps your clients from re-creating the stampede, and a few guardrails ensure you retry only what should be retried. Spend an afternoon adding these to your HTTP clients and background jobs, and you'll turn one of the most common sources of cascading failure into a quiet non-event.