> ## Documentation Index
> Fetch the complete documentation index at: https://docs.omnia-voice.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Online monitoring & the quality-gated canary

> Score a sample of live traffic continuously, alert on interval bounds, and let a gate promote or roll back a model change on statistical evidence.

**Monitor online** (per judge) scores a deterministic sample of fresh
logged traffic continuously: capped per tick, billed as normal usage, and
never double-judging a request. Trace-unit judges monitor **completed
runs**: a trace quiet for 10 minutes is judged once, as a whole trajectory,
keyed to the trace so a straggler step never causes a re-judge. The rolling
corrected pass rate feeds:

* **`quality_low` alerts**: fired when the rate's CI **lower bound** crosses
  your floor with enough judgments to mean it. A bad hour on 8 samples is
  noise, not an incident.
* **The quality-gated canary**: set `gate_*` fields on a
  [model alias](/reference/model-aliases) canary split and the online scorer
  judges *both arms* of the live split separately. The gate:
  * **holds** until both arms have `gate_min_samples` scored requests,
  * **promotes** only when the canary's corrected-rate CI lower bound is at
    least the incumbent's: demonstrably not-worse, not point-estimate lucky,
  * **rolls back** only when even the canary's *optimistic* bound sits below
    your rollback threshold: confidently bad, not unluckily sampled.

Every gate lives on the **Live switches** tab (arms, sample counts, and the
latest verdict) and the same gate config is editable **in the alias dialog
on the Models page**, not only via the API.

**Trace-quality gating**: point `gate_criterion_id` at a trace-unit
judge and the gate scores completed runs of the alias's traffic: a model
swap evaluated on whole agent runs or conversations. Arm attribution is
honest or absent: a run whose steps touched both models counts toward
neither.

**Monitor config**: everything in the Monitor dialog (`online_enabled`,
`online_percent`, the population (tag and segment), and the weekly spend
ceiling `online_cap_usd`, 0 = uncapped) is equally settable via
`PATCH /v1/criteria/{id}`, same validation, no forked rules. When a judge's
online judging bill reaches its cap, that judge pauses until the week rolls
over, and everything else keeps running. Every billed micro-cent counts,
judge retries included. Changing the cap never voids calibration (money
config is not instrument config); changing the population does.

`gate_mode: "recommend"` (default) surfaces verdicts on `GET /v1/aliases`
and via your alert email/webhook; you click the button. `"auto"` lets the
gate repoint the alias itself, always audited with the numbers, and requires
a **calibrated** judge: a judge with a weaker signal may recommend, but it
cannot repoint traffic itself.

```bash theme={null}
curl -X PUT "https://gateway.omnia-voice.com/v1/aliases" -H "Authorization: Bearer $OMNIA_API_KEY" \
  -H "Content-Type: application/json" -d '{
    "name": "support-bot",
    "target_model": "Qwen/Qwen3-32B",
    "canary_model": "Qwen/Qwen3-30B-A3B-Instruct-2507",
    "canary_percent": 10,
    "gate_criterion_id": "crit_...",
    "gate_mode": "recommend",
    "gate_min_samples": 50
  }'
```

## Drift detection

Corrected pass rates rely on the judge's TPR/TNR staying true on live traffic.
When your traffic shifts (a release, a new failure mode, a change in what
users ask), a judge that was calibrated last month may quietly drift, and a
stale correction would report confident numbers that are wrong.

Omnia watches for this. A background sweep flags a judge whose calibration
has gone **stale** or whose live corrected-quality has dropped anomalously; the
flagged judge shows `drift_status: "flagged"`, and the review queue prompts
you to **re-calibrate** it on a fresh slice of grades, recomputing a current,
localized TPR/TNR. The instrument that catches drift in *your* models is held
to the same standard itself: a validated number that goes stale gets caught and
re-measured, not trusted indefinitely.

## Design rules

* **Binary verdicts everywhere.** Pass/fail aligns with humans; 1–5 scores
  don't.
* **Judges rank and monitor in aggregate; they never gate a single request.**
  Inline guardrails are deterministic; evaluators are sampled and
  statistical.
* **Alert and promote on interval bounds, never point estimates.**
* **Partial responses are review signals, never data.** Aborted streams are
  visible in the review queue and Logs (opt-in view) but structurally
  excluded from datasets, exports, comparison sampling, and online scoring.
* **The human stays the judge.** Suggestions, scans, and auto-improve amplify
  your grades; nothing auto-grades, and only a calibrated judge in explicit
  auto mode may touch routing.