Skip to main content
POST
Align a criterion against your labels

Authorizations

Authorization
string
header
required

Your workspace API key, e.g. sk_sovereign_..., sent as Authorization: Bearer <key>.

Path Parameters

id
string
required

Response

Label sets of 50 or fewer are judged synchronously and return the AlignmentReport. Larger sets return AlignmentQueued ({queued: true, total_labels}) and a background worker builds the report over the following minutes; poll GET /v1/criteria/{id} for the metrics and GET /v1/criteria/{id}/alignment for the full report.

Returned by POST /v1/criteria/{id}/align for label sets of 50 or fewer (larger sets return AlignmentQueued and the report is built over the following minutes).

scope_tag
string

The population measured — the criterion's tag ('' = all labeled traffic). Alignment numbers only describe the population they were measured on.

scope_family
string

The auto-detected traffic segment measured ("" = no segment scope). The second population handle, same binding rule as scope_tag.

tag_breakdown
object[]

What the measured labels were made of, by tag ('' = untagged). One entry for a scoped criterion; the pollution report for an unscoped one.

mixed_population
boolean

True when an UNSCOPED criterion was measured across more than one tag — the metrics describe a blend of tasks, not any one of them. Scope the criterion and re-align.

excluded_other_cause
integer

Fails attributed to other criteria: in scope but excluded from this criterion's calibration set by cause attribution.

unattributed_fails
integer

In-population fails with no cause attribution. They still count as fails for every criterion; attributing them tightens narrow judges' numbers.

metrics
object

n, tpr, tnr, kappa and the 95% Wilson intervals (tprCi, tnrCi; camelCase on this payload). tpr/tnr are null when the label set has no human fails/passes to measure against.

tier
enum<string>
Available options:
aligned,
weak,
misaligned,
unmeasured
thin_alignment_set
boolean

True under 100 judgeable labels: intervals stay wide; label more.

skipped
integer

Labeled traces whose content aged out, or whose judge verdict was unparseable twice.

holdout
object | null

Present at 80+ judged labels: {tune_n, report_n}. Metrics above come from the report half only; tune-half rows exist for prompt iteration.

one_class_note
string | null

Set when null TPR/TNR is a one-class measured set (a labeling gap), naming the missing class, so it is never mistaken for "never aligned" or a broken judge.

disagreements
object[]