Self-training rounds (offline)
For each logged prompt, the platform samples several candidate answers,
keeps the ones your judge passes, and fine-tunes on the winners. Prompts
that fail every attempt are set aside rather than forced.
Online RL runs (GRPO)
The prompts self-training gave up on become the candidate queue, and
online reinforcement learning trains directly against your judge’s graded
signal on exactly those prompts, where offline sampling can’t help.
Why the judge must be aligned
Training a model toward an evaluator’s opinion is only safe when the evaluator is measured. Omnia refuses to start any self-improvement work unless the judge is aligned: validated against your human labels with true-positive and true-negative rates at or above the platform bar. An unvalidated judge as a training signal is reward hacking by construction; the platform will not do it.Safety rails on every run
- Reward budget: a hard ceiling on judge spend, enforced by the platform’s ledger (never self-reported by the trainer). A run that hits its ceiling is refused further scoring and marked over-budget.
- GPU-hour budget: a wall-clock ceiling on the training side.
- Hacking tripwire: a run whose policy collapses onto degenerate judge-pleasing answers (entropy collapse, vanishing reward variance, drifting divergence) is stopped automatically and flagged.
- Held-out gate: a slice of prompts is reserved at run start that the trainer never sees; adoption decisions use held-out judge pass-rate, not training reward.
- Stop button: stopping a run takes effect at its next scoring call.