Gate a deploy on an eval run
One curl in CI: 200 when every requested check passes, 412 otherwise — curl -f does the rest.
Statistical thresholds gate on confidence-interval lower bounds, never point estimates: a 62% win rate whose interval reaches down to 51% does not clear a 55% bar. Assertion rates are deterministic and gate exactly. A run that isn’t DONE fails closed — poll until you get a verdict.
curl -f "$BASE/v1/evals/$RUN/gate?min_assertion_pass_rate=1&min_win_rate=0.5" \
-H "Authorization: Bearer $OMNIA_API_KEY" || exit 1
Authorizations
Your workspace API key, e.g. sk_sovereign_..., sent as Authorization: Bearer <key>.
Path Parameters
Query Parameters
Comparison runs: every candidate's win-rate CI lower bound must clear this.
0 <= x <= 1Criterion runs: every model's corrected pass-rate CI lower bound must clear this (falls back to the observed interval when the judge was never aligned — the check notes it).
0 <= x <= 1Every model's all-assertions pass rate (exact) must clear this.
0 <= x <= 1Restrict the checks to one model.
The certified switch test: on a criterion run whose baseline was the incumbent's stored production answers, each candidate's corrected pass-rate CI lower bound must reach the incumbent arm's corrected rate minus this margin (0.05 = provably within 5 points at worst). Requires corrected rates on both arms, which only a calibrated judge produces; there is no observed-rate fallback, so an uncalibrated judge cannot certify a switch.
0 <= x <= 1