Sidequest: Why Retries Wait Longer Each Time

Cold open

Backoff means that when something fails and you try again, you wait a little longer before each new try instead of hammering away at a fixed pace.

That is the whole idea. The interesting part is why the waiting has to grow.

How this question got here

This one has an odd backstory. "Backoff, why wait longer each retry" was the note you typed for the last acceptance test of the huh mod. The session had just fixed things so that your slash huh note would stop leaking into Claude's context. The point of that test was that Claude should not be able to read your question. It could not, and that is why the question reaches you as an episode and not as an answer in the chat. The privacy fix worked, so the answer comes on a walk instead of in the terminal.

The excerpt does not show where you first ran into the word, so here is the general idea, followed by where it lives in your own estate.

The mental model

A failure can be one of two kinds. Some failures are blips: a network hiccup, a server busy for a second, a preflight call that timed out once. Others are real outages, where the thing on the other end is down and will stay down for a while.

When the first try fails, you do not know which kind you have. Backoff is a cheap way to bet on both.

The first retry comes quickly, because most failures are blips, and a quick retry clears a blip before anyone notices. If that retry fails too, the evidence has shifted. A blip would probably have cleared by now, so this is starting to look like a real outage. Retrying every few seconds against a real outage does nothing useful, so you wait longer. If that fails as well, you wait longer again. Each wait matches what you now believe about the failure.

There are two reasons not to keep retrying at a fixed fast pace.

The first is that it hurts. A service that is struggling, maybe overloaded, gets hit even harder by everyone who is retrying it. Picture a thousand clients that all failed at the same moment and all retry every second. They are no longer waiting for the service to recover. They are the reason it cannot. Engineers call this a thundering herd. Big platforms add a bit of randomness to each wait, called jitter, so the clients drift apart instead of charging back in step. Even the largest cloud providers have taken themselves down this way at least once, which is comforting in a petty sort of way.

The second is that it is wasteful. Every attempt costs something: tokens, an SMS, a slot on the usage meter, a line in the log. Fifty attempts during an hour-long outage buy you nothing that 3 well-spaced attempts would not.

The half people forget: the deadline

Backoff on its own just retries more and more slowly forever. That is a failure that never admits it is a failure. So the Assistant's spec does not say "backoff". It says "backoff with deadline". The waits grow, but there is also a fixed window, and when the window closes, the job stops retrying and reports a failure that a human sees.

This came out of autocast's history. In the old version, the thing everyone called a "cooldown" was really a hold that retried on every tick until a maximum number of days had passed. That arithmetic was the single most complained-about defect in autocast: for a daily show, the give-up condition never fired. When the Assistant took over, the spec renamed the primitive and made the deadline part of it.

Where it lives in your estate

The clearest example is the Director Report, the Sunday evening show. It is the only one marked critical, and it is the only one that has a retry budget. Its registry entry lists 2 waits, 15 minutes and then 1 hour, inside a fixed window of 2 days measured from the first failure. That means 1 initial attempt and 2 retries, with growing gaps, and a hard edge after that. The Assistant also refuses to load a config where the waits add up to more than the window, because a retry scheduled after its own deadline makes no sense.

This was added after a weekly episode was lost in late August. A preflight check timed out once, and the whole week's report disappeared because nothing tried again.

One more rule makes the design honest. Only a conclusive failure is retried. If the outcome is unknown, for example if the job might have published before it crashed, the Assistant never retries, because retrying could publish twice. Backoff is for "it definitely did not work", not for "I am not sure what happened". That is the same caution as the claim ticket from an earlier episode: retries are explicit authority, not a reflex.

The ordinary shows have no retry list at all. If a usage poll fails, the next tick simply runs it again 5 minutes later. That is the right choice when the job is cheap and the tick already gives you a natural, polite pace.

If you remember one thing: wait a little after the first failure because it is probably a blip, wait longer after each one after that because it probably is not, and always set a deadline after which the system stops trying and tells a human.