Skip to main content
Validated evals give you a judge you can trust. Self-improvement uses that judge as a reward: Omnia samples answers from your model on your own traffic, keeps only the ones the validated judge passes, and trains on the winners. The result is a model measurably better at your bar, and the proof is the same holdout eval that validated the judge in the first place. This is the offline form of reinforcement learning (RAFT: reward-weighted rejection-sampling). It reaches the same objective as online RL, preferring high-reward answers, without an online trainer, which is why it runs today on infrastructure you already use.
Self-improvement is only as trustworthy as its reward. Omnia refuses to train against an uncalibrated judge: the reward must be a calibrated judge (an aligned criterion in the API), or a compositional matrix of them (below). An uncalibrated reward trains a model toward the judge’s mistakes; the gate exists to prevent exactly that.

​
How a run works

1

Pick a reward

A calibrated judge (TPR and TNR ≥ 0.9), or a compositional reward — several weakly calibrated (≥ 0.75) judges plus deterministic assertions, all required to pass. See Choosing a reward.
2

Sample and score

For each prompt drawn from your logged traffic (tag-scoped), Omnia generates N candidate answers (default 4, up to 8) at temperature, then scores every candidate with the reward. Rollout and judge calls bill as normal usage.
3

Keep the winners

The single best judge-passed answer per prompt becomes one training example. Prompts where no candidate passes are dropped; that filtering is the optimization. A run needs at least 10 winning prompts; below that, training would overfit noise, and the round stops with a clear reason.
4

Train

The winners become a supervised fine-tune dataset and a training job on the base model, priced and wallet-gated like any fine-tune. Long jobs run durably (they are not bound to a request timeout).
5

Prove it

The trained model is not trusted on faith. Bake it off against the base on your holdout grades, with the same calibrated judge. If the corrected pass rate improved on the holdout, the run worked; if not, it didn’t, and the number tells you which. See Corrected pass rates.

​
Choosing a reward

A single calibrated judge is the simplest reward, but a clean 0.9 calibration is hard to reach for subjective, long-horizon work: an open-ended support agent, a negotiation, a multi-step task. For those, a compositional reward lets you train without a single judge that clears the full bar:
  • Combine several judges each at the weakly calibrated tier (≥ 0.75) with deterministic assertions (valid JSON, a required tool was called, a format held).
  • A candidate is a winner only if every judge and every assertion passes.
Requiring multiple independent judgments to agree is not a weaker bar. It is a harder one to game. A model can learn to fool one judge; fooling three orthogonal judges and two exact checks at once is far less likely, which makes the compositional reward more resistant to reward-hacking, not less.
Deterministic assertions cost nothing and cannot be gamed: lean on them. Every assertion you can express (schema, tool-called, length, regex) is a verifiable reward that tightens the bar for free.

​
Tuning a run

  • Candidates per prompt (N): more samples per prompt means more chances to clear the reward, so a strict judge yields more winners at higher N. Raise it when a run stops short of the winner floor. Cost scales with N.
  • Population: the tag whose traffic you sample from, the same population the reward judge is calibrated against. Training on support traffic with a support-calibrated judge; never mixing the two.
  • When a run comes back thin: “only 4 prompts produced a judge-passed answer” is a real signal, not a failure: the current model rarely clears this bar. Raise N, ease the reward, or accept that this is a genuinely hard task where the lift will be large once training lands.

​
Deploying the result

A self-improved model is a fine-tune like any other. Deploy it to a dedicated endpoint and call it through the same API as any model, put it behind a model alias and let the quality-gated canary promote it on live traffic only if it holds up, or take the weights and run them yourself. See Owning your model.

​
Why this is safe

Every guard in validated evals carries into training:
  • The reward is validated: calibrated judges only; uncalibrated judges are refused.
  • The proof is held out: the trained model is measured on grades the judge never tuned against, so an improved number can’t be a fit.
  • The gate is statistical: promotion to production runs through the same CI-lower-bound canary as any model change.
  • Money is metered: rollout, judging, and the training job all bill and wallet-gate exactly like normal usage; nothing trains or ships silently.
To be clear about what this is: self-improvement is not “point at your logs and get a better model.” It is “prove a judge, use it as a reward, and prove the result against the same bar”: a process where every step is measured, and the number at the end tells you whether it worked.