> ## Documentation Index
> Fetch the complete documentation index at: https://docs.omnia-voice.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate limits

> How throughput limits work and how to handle them.

Every model has per-model throughput limits: a **tokens-per-minute (TPM)** limit
and a **requests-per-minute (RPM)** limit, synced from the provider catalog.
When your traffic exceeds a limit, Omnia returns `429`.

<Warning>
  Catalog numbers are the upstream provider's advertised capacity, **not a
  throughput guarantee on the shared gateway**. In our own sustained-load
  measurements, real end-to-end throughput tops out well below the largest
  catalog figures (on the order of \~300 requests/minute sustained per workspace,
  as of August 2026); latency, upstream queuing, and shared capacity all bind
  first. Size your integration against what you measure, not the catalog column.
  For sustained high volume, use a [dedicated endpoint](/dedicated/overview):
  the GPU is reserved for you, so its ceiling is yours.
</Warning>

<Note>
  Limits are **per model**, not a single global ceiling. Spreading load across
  models also adds headroom.
</Note>

## Handling 429s

When you receive a `429`, back off and retry with exponential backoff **and
jitter**. Jitter spreads retries out over time so a fleet of clients doesn't all
retry in lockstep and stampede the limit again.

<Steps>
  <Step title="Catch the 429">
    Detect the `429` status (or a `RateLimitError` from your client library).
  </Step>

  <Step title="Wait, backing off exponentially">
    Sleep for an interval that doubles each attempt, capped at a ceiling
    (e.g. 30s), and add a small random jitter on top.
  </Step>

  <Step title="Retry, then give up">
    Retry up to a fixed number of attempts. If you're still limited after the
    last attempt, surface the error rather than looping forever.
  </Step>
</Steps>

```python theme={null}
import time
import random

def with_backoff(fn, retries=5):
    for attempt in range(retries):
        try:
            return fn()
        except RateLimitError:  # your client's 429 exception
            if attempt == retries - 1:
                raise
            # Exponential backoff capped at 30s, plus jitter.
            time.sleep(min(30, 2 ** attempt) + random.random())
```

<Tip>
  Because completions are only billed for tokens actually produced, a retried
  request that was rate-limited before it ran costs you nothing. Retrying `429`s is
  safe on the wallet.
</Tip>

## Reducing pressure

If you're regularly hitting `429`, reduce the load you put on a model's limits
rather than just retrying harder.

<CardGroup cols={2}>
  <Card title="Batch embeddings" icon="layer-group">
    Send an array of inputs in a single embeddings request instead of one
    request per input. Fewer requests means less RPM pressure for the same work.
  </Card>

  <Card title="Cap output" icon="scissors">
    Set `max_tokens` to what you actually need. Shorter outputs consume less of
    the TPM budget.
  </Card>

  <Card title="Limit concurrency" icon="sliders">
    Cap how many requests you fire in parallel from a single workspace so you
    stay under the RPM limit instead of bursting past it.
  </Card>

  <Card title="Dedicated capacity" icon="server" href="/dedicated/overview">
    For sustained high volume, a dedicated endpoint gives you reserved
    throughput instead of sharing a shared model's limits.
  </Card>
</CardGroup>

## Checking headroom

You don't have to wait for a `429` to know you're approaching a limit.
Rate-limit **headroom**, how close your traffic is running to a model's limits,
is visible in your [observability](/reference/observability). Each request's
detail in the request log shows its headroom, so you can watch how much of a
model's TPM and RPM budget you're consuming and scale back (or move to a
dedicated endpoint) before you start hitting limits.