Label a trace
Record a human pass/fail verdict on one logged request (owner/admin key — labels DEFINE quality). Downstream systems that already know an output was good or bad can push verdicts here too: a support tool marking an escalation, a pipeline that caught a wrong extraction.
Labels are the foundation of validated evals: judge alignment (TPR/TNR/κ), corrected pass rates, and the quality-gated canary all hang off them. Aim for ~100; stop when 20 in a row teach you nothing new.
Authorizations
Your workspace API key, e.g. sk_sovereign_..., sent as Authorization: Bearer <key>.
Body
Labels created over the API are request-scoped (one exchange). Whole-run (trace) grading is done in the dashboard's review queue.
Binary, always. Upsert by request_id: re-labeling replaces the verdict — the newest judgment is the truth.
pass, fail WHY, in your words (max 2000 chars). Strongly encouraged on fails: critiques become judge few-shot material, feed suggested judges, and make disagreement review meaningful.
2000Response
The stored label.
The gateway request this verdict is about.
pass, fail WHY it failed — becomes judge few-shot material and the raw text 'Suggest criteria' clusters into your failure taxonomy.
human, assist_accepted request grades one exchange; trace grades a whole agent run (the label rides the run's final step). Labels created over this API are request-scoped.
request, trace Cause attribution: criterion ids this failure indicts. Managed from the dashboard's review queue; empty on unattributed fails.