Deploying a fine-tuned model

A fine-tuned model isn’t served on the shared per-token fleet. A completed job produces a model artifact, and you serve it by creating a dedicated endpoint bound to that artifact, then calling it by the endpoint’s routing key.

Evaluate before you deploy

You don’t have to deploy to find out whether the training worked. A bake-off (POST /v1/fine_tuning/jobs/{id}/bakeoff, or the button on the job page) compares the tuned model against its base on the dataset’s held-out split: token-level NLL and perplexity on the reference answers, plus, if you pass judge_criterion_id, a calibrated judge’s pass rate on both sides. The verdict is sign-test-backed: it reads improved or regressed only when the win count is statistically significant, and inconclusive otherwise. Poll GET .../bakeoff for status, verdict, and ledger-true spent_usd. A job whose dataset kept no holdout can’t be baked off, and the response says so instead of improvising a number.

Track deploy progress

Deploying is a multi-stage pipeline, and GET /v1/fine_tuning/jobs/{id} reports where yours is: deploy_status moves through queued → staging → relaying → converting → provisioning → serving (the platform stages the trained weights, converts them into a servable artifact, and provisions the endpoint), or lands on failed with a deploy_error naming the reason. deployed_model_name is set once the model is callable, and stays null until then. Poll it instead of guessing.

Bind the model when creating an endpoint

Create a dedicated endpoint (POST /v1/dedicated) and point it at your trained model using one of two fields:

customWeightsId: the model artifact id produced by the job.
fineTuningJobId: a completed fine-tuning job, whose output artifact is served.

curl https://gateway.omnia-voice.com/v1/dedicated \
  -H "Authorization: Bearer $OMNIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-tone-serve",
    "modelName": "meta-llama/Llama-3.1-8B-Instruct",
    "fineTuningJobId": "<job-id>",
    "gpuType": "<gpu-type>",
    "gpuCount": 1,
    "region": "<region>",
    "minReplicas": 1,
    "maxReplicas": 2
  }'

import requests

BASE = "https://gateway.omnia-voice.com/v1"
headers = {
    "Authorization": f"Bearer {OMNIA_API_KEY}",
    "Content-Type": "application/json",
}

# Serve by completed job id...
endpoint = requests.post(
    f"{BASE}/dedicated",
    headers=headers,
    json={
        "name": "support-tone-serve",
        "modelName": "meta-llama/Llama-3.1-8B-Instruct",
        "fineTuningJobId": "<job-id>",
        "gpuType": "<gpu-type>",
        "gpuCount": 1,
        "region": "<region>",
        "minReplicas": 1,
        "maxReplicas": 2,
    },
).json()

# ...or by the model artifact id:
#   "customWeightsId": "<artifact-id>"

The create call returns just the new endpoint’s id; GET /v1/dedicated/{id} to read its routing key and status once it’s ready. See Deploying an endpoint for the full field reference and how to discover available GPU types and regions.

Deploy from the dashboard

Once a fine-tuning job succeeds, open it in the dashboard and choose Deploy to serve. Pick the GPU type and region, and Omnia provisions a dedicated endpoint serving your trained weights, reusing the same deploy gate, minimum runway, and per-minute metering as any dedicated endpoint.

Calling your model

After deployment, call it like any dedicated endpoint: by its routing key, through the standard inference API:

curl https://gateway.omnia-voice.com/v1/chat/completions \
  -H "Authorization: Bearer $OMNIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dedicated/<your-routing-key>",
    "messages": [{"role": "user", "content": "..."}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.omnia-voice.com/v1",
    api_key="$OMNIA_API_KEY",
)

resp = client.chat.completions.create(
    model="dedicated/<your-routing-key>",
    messages=[{"role": "user", "content": "..."}],
)

Billing

Fine-tuned models are served on dedicated capacity, so serving is billed per GPU-hour (metered per-minute) while the endpoint runs, not per token. This is separate from the one-time per-trained-token charge for the training run itself. Disable the endpoint when you don’t need it to halt billing.

Your trained model is private to your workspace. Deploying it does not expose it to other tenants.

Managing it

Scaling, disabling, and deleting work the same as any dedicated endpoint; see Managing endpoints.

Adopt through a model alias

Every completed training round also mints an immutable model version (GET /v1/model_versions), pinned to the judge, curriculum, and holdout its verdict depended on. Once the version is serving, POST /v1/model_versions/{id}/adopt with {"aliasName": "support-bot"} points your alias at it: production traffic moves with no client change, the repoint is audited, and rolling back is the same call aimed at an older version.

Dedicated deploy reference

Full create-endpoint field reference.

Manage endpoints

Call, scale, disable, and delete.

Get started

Capture

Observe

Grade & Judge

Evaluate

Improve

Run at scale

Platform

Deploying a fine-tuned model

Evaluate before you deploy

Track deploy progress

Bind the model when creating an endpoint

Deploy from the dashboard

Calling your model

Billing

Managing it

Adopt through a model alias

Dedicated deploy reference

Manage endpoints

​Evaluate before you deploy

​Track deploy progress

​Bind the model when creating an endpoint

​Deploy from the dashboard

​Calling your model

​Billing

​Managing it

​Adopt through a model alias

Dedicated deploy reference

Manage endpoints

Evaluate before you deploy

Track deploy progress

Bind the model when creating an endpoint

Deploy from the dashboard

Calling your model

Billing

Managing it

Adopt through a model alias