> ## Documentation Index
> Fetch the complete documentation index at: https://docs.omnia-voice.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Self-improvement

> Turn a validated judge into a better model: reward-weighted training on your own traffic, with the same judge as the reward and the same holdout as the proof.

Validated evals give you a judge you can trust. Self-improvement uses that
judge as a **reward**: Omnia samples answers from your model on your own
traffic, keeps only the ones the validated judge passes, and trains on the
winners. The result is a model measurably better *at your bar*, and the proof
is the same holdout eval that validated the judge in the first place.

This is the offline form of reinforcement learning (RAFT: reward-weighted
rejection-sampling). It reaches the same objective as online RL, preferring
high-reward answers, without an online trainer, which is why it runs today on
infrastructure you already use.

<Note>
  Self-improvement is only as trustworthy as its reward. Omnia **refuses to
  train against an uncalibrated judge**: the reward must be a
  [calibrated judge](/judges/calibration) (an *aligned criterion* in the
  API), or a compositional matrix of them (below). An uncalibrated reward
  trains a model toward the judge's mistakes; the gate exists to prevent
  exactly that.
</Note>

## How a run works

<Steps>
  <Step title="Pick a reward">
    A **calibrated** judge (TPR and TNR ≥ 0.9), or a **compositional reward**
    — several weakly calibrated (≥ 0.75) judges plus deterministic
    assertions, all required to pass. See
    [Choosing a reward](#choosing-a-reward).
  </Step>

  <Step title="Sample and score">
    For each prompt drawn from your logged traffic (tag-scoped), Omnia generates
    N candidate answers (default 4, up to 8) at temperature, then scores every
    candidate with the reward. Rollout and judge calls bill as normal usage.
  </Step>

  <Step title="Keep the winners">
    The single best judge-passed answer per prompt becomes one training example.
    Prompts where no candidate passes are dropped; that filtering *is* the
    optimization. A run needs at least 10 winning prompts; below that, training
    would overfit noise, and the round stops with a clear reason.
  </Step>

  <Step title="Train">
    The winners become a supervised fine-tune dataset and a training job on the
    base model, priced and wallet-gated like any fine-tune. Long jobs run
    durably (they are not bound to a request timeout).
  </Step>

  <Step title="Prove it">
    The trained model is **not** trusted on faith. Bake it off against the base
    on your holdout grades, with the same calibrated judge. If the corrected pass
    rate improved on the holdout, the run worked; if not, it didn't, and the
    number tells you which. See [Corrected pass rates](/evaluate/corrected-rates).
  </Step>
</Steps>

## Choosing a reward

A single calibrated judge is the simplest reward, but a clean 0.9 calibration
is hard to reach for subjective, long-horizon work: an open-ended support
agent, a negotiation, a multi-step task. For those, a **compositional reward**
lets you train without a single judge that clears the full bar:

* Combine several judges each at the **weakly calibrated** tier (≥ 0.75) with
  deterministic **assertions** (valid JSON, a required tool was called, a
  format held).
* A candidate is a **winner only if every judge and every assertion
  passes**.

Requiring multiple independent judgments to agree is not a weaker bar. It is
a *harder* one to game. A model can learn to fool one judge; fooling three
orthogonal judges and two exact checks at once is far less likely, which
makes the compositional reward more resistant to reward-hacking, not less.

<Note>
  Deterministic assertions cost nothing and cannot be gamed: lean on them.
  Every assertion you can express (schema, tool-called, length, regex) is a
  verifiable reward that tightens the bar for free.
</Note>

## Tuning a run

* **Candidates per prompt (N)**: more samples per prompt means more chances to
  clear the reward, so a strict judge yields more winners at higher N. Raise it
  when a run stops short of the winner floor. Cost scales with N.
* **Population**: the tag whose traffic you sample from, the same population
  the reward judge is calibrated against. Training on support traffic with a
  support-calibrated judge; never mixing the two.
* **When a run comes back thin**: "only 4 prompts produced a judge-passed
  answer" is a real signal, not a failure: the current model rarely clears this
  bar. Raise N, ease the reward, or accept that this is a genuinely hard task
  where the lift will be large once training lands.

## Deploying the result

A self-improved model is a fine-tune like any other. Deploy it to a
[dedicated endpoint](/dedicated/overview) and call it through the same API as
any model, put it behind a [model alias](/reference/model-aliases) and let the
[quality-gated canary](/judges/online-monitoring)
promote it on live traffic only if it holds up, or take the weights and run
them yourself. See [Owning your model](/reference/own-your-model).

## Why this is safe

Every guard in validated evals carries into training:

* **The reward is validated**: calibrated judges only; uncalibrated judges
  are refused.
* **The proof is held out**: the trained model is measured on grades the
  judge never tuned against, so an improved number can't be a fit.
* **The gate is statistical**: promotion to production runs through the same
  CI-lower-bound canary as any model change.
* **Money is metered**: rollout, judging, and the training job all bill and
  wallet-gate exactly like normal usage; nothing trains or ships silently.

To be clear about what this is: self-improvement is not "point at your logs
and get a better model." It is "prove a judge, use it as a reward, and prove
the result against the same bar": a process where every step is measured, and
the number at the end tells you whether it worked.