Requests per minute, tokens per minute and concurrent sessions are three different ceilings, and running several agents in parallel hits all of them at the account level even though each agent feels like an independent process. How to handle a 429 mid-tool-call, and how to size a fleet before the ceiling finds you mid-run instead of after.
three different limits, doing different jobs
"Rate limit" gets used as one term for at least three separate ceilings, and a coding agent setup can hit any of them independently. Knowing which one you tripped changes what you do about it.
- Requests per minute (RPM). A cap on how many API calls you can make in a window, regardless of how big each call is. A chatty agent that makes many small tool calls can hit this even with a short context.
- Tokens per minute (TPM). A cap on the total input and output tokens across those calls in the same window. Coding agents typically resend the growing conversation history on every turn, so TPM tends to be the first ceiling a long session finds, well before RPM does.
- Concurrent sessions or connections. A cap on how many requests can be in flight at once, independent of rate. This is the one that surprises people running agents in parallel, because it is not about speed at all, only about how many calls are simultaneously open.
Providers publish these as tiers that typically scale with usage history and billing standing, and the exact numbers change over time and differ by account. Nothing in this post should be read as a claim about what any specific tier is set to today; check your own provider console or the rate-limit headers on a live response for the number that actually applies to you.
handling a 429 in the middle of a tool call
The failure mode worth designing for specifically is a 429 arriving after an agent has already done real work: it ran a tool, produced a result, and the call that was supposed to report that result back to the model gets rate limited. The wrong response is to treat this as the turn failing and move on, because that silently discards work that already happened, a command that already ran, a file that already changed, a test that already executed. The agent's next turn then either repeats the side effect or proceeds without knowledge of it, both of which are worse than waiting a few seconds.
The right response is ordinary backoff applied to the specific call, not the whole session:
- Honor Retry-After when it is present. Most providers tell you how long to wait. Guessing shorter than that just spends the wait retrying into more 429s.
- Fall back to exponential backoff with jitter when it is absent. Fixed-interval retries from several agents on the same key tend to resynchronize and hit the provider in the same instant repeatedly, which is the standard argument for adding randomness to the wait.
- Resend the specific rejected call, not the whole conversation from scratch. The tool result computed locally is still valid; only the network call reporting it needs to be retried.
- Cap the retries and surface the wait, rather than looping silently. An agent stuck retrying for minutes with nothing printed anywhere looks identical to a hung agent. A visible "waiting on rate limit, retrying in Ns" is cheap and saves someone from killing a session that was about to succeed.
async function callWithBackoff(fn, { maxRetries = 5 } = {}) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
return await fn()
} catch (err) {
if (err.status !== 429 || attempt === maxRetries) throw err
const retryAfter = Number(err.headers?.['retry-after'])
const wait = Number.isFinite(retryAfter)
? retryAfter * 1000
: Math.min(30_000, 500 * 2 ** attempt) + Math.random() * 250
await new Promise(r => setTimeout(r, wait))
}
}
}What this does not fix
Backoff makes a single rejected call eventually succeed. It does nothing about a fleet that is structurally sized past the ceiling, where every agent is perpetually backing off and total throughput quietly drops well below what one well-behaved agent alone would achieve. Backoff is a correctness fix for one call. Fleet size is a capacity planning problem, covered next.
sizing a fleet around a ceiling instead of discovering it mid-run
The cheaper version of the 429 problem is not hitting it in the first place, which is a planning question rather than a retry-logic question: how many agents can this account actually run at once, given the limits it currently has.
As a rule of thumb rather than a formula that will be exactly right for any given workload: look up your account's current TPM limit for the model you are using, estimate the steady-state tokens per minute one agent burns once its context has grown to a typical working size, and divide, leaving deliberate headroom below the quotient rather than aiming at it. A fleet running at 100% of a shared ceiling has no room for one agent having a verbose turn without everyone else stalling.
- Read the limit from the source, not from memory or a blog post. Provider dashboards and response headers report the number that applies to your account right now; anything written down elsewhere, including in this post, ages the moment it is published.
- Estimate per-agent usage from the workload, not from the first few turns. A coding agent's context grows over a session, so its tokens-per-minute rate late in a long task is higher than its rate in the first exchange. Size against the late-session rate.
- Leave headroom, don't spend it all. Something like 70 to 80% of the ceiling as a target for steady-state fleet usage is a defensible rule of thumb, not a measured optimum, and it exists specifically to absorb the agent that has an unusually verbose turn without taking the whole fleet into backoff.
- Re-check after any tier change. Limits commonly step up with usage history and billing standing. A fleet size chosen against last month's ceiling may be leaving capacity on the table, or may already be past a ceiling that moved the other way.
what to actually watch while a fleet runs
Planning ahead of time reduces how often you hit the ceiling. It does not remove the need to watch while the fleet is running, because the estimate that sized the fleet is exactly that, an estimate.
- Remaining-limit headers, aggregated across agents. Any one agent's headers show the account-wide remaining budget, since the limit is shared. Watching one agent's headers tells you about all of them.
- 429 rate over time, not just its presence. An occasional 429 that backoff absorbs cleanly is normal operation under load. A rising rate of them is the fleet telling you it is oversized for the current ceiling before throughput visibly drops.
- Which agent is actually blocked versus which is just quiet. An agent waiting out a backoff and an agent that finished its turn and is idle look the same in a terminal you are not watching closely. The distinction matters because one needs nothing from you and the other might.
common questions
What is the difference between requests-per-minute and tokens-per-minute limits?
RPM caps how many API calls you can make in a minute regardless of size, while TPM caps the total tokens across those calls in that window. A large context window can exhaust a TPM limit in a handful of requests well before RPM becomes the binding constraint, which is typical for coding agents that resend a growing conversation on every turn.
Why do multiple parallel coding agents share one rate limit?
Because the limit is enforced against the account or API key making the calls, not against the process making them. Five agent processes calling the same key are five callers sharing one bucket, the same way five browser tabs open to the same site share one connection quota at the server.
What should an agent do when it gets a 429 in the middle of a tool call?
Back off and retry the specific call that was rejected, honoring any Retry-After the response provides, rather than treating the 429 as a failure of the whole turn. The tool result that was already computed locally should not be discarded; only the API call reporting it needs to be resent.
How many agents can I run in parallel before hitting a rate limit?
There is no universal number because limits vary by provider, tier and change over time, so the right approach is to read your own limit from the provider dashboard or response headers and divide by your per-agent tokens-per-minute rate, leaving headroom rather than assuming your process count is the constraint.