Chat completions

Chat completions are the core of Omnia. The endpoint is fully OpenAI-compatible, so any OpenAI SDK or HTTP client works by pointing base_url at https://gateway.omnia-voice.com/v1 and using your Omnia API key. Request and response shapes are the standard OpenAI shapes: id, object, choices, and a usage object with prompt_tokens, completion_tokens, and total_tokens.

Basic request

curl https://gateway.omnia-voice.com/v1/chat/completions \
  -H "Authorization: Bearer $OMNIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-32B",
    "messages": [
      { "role": "system", "content": "You are a concise assistant." },
      { "role": "user", "content": "Summarize the water cycle in one sentence." }
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.omnia-voice.com/v1",
    api_key="OMNIA_API_KEY",
)

resp = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Summarize the water cycle in one sentence."},
    ],
)

print(resp.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.omnia-voice.com/v1",
  apiKey: process.env.OMNIA_API_KEY,
});

const resp = await client.chat.completions.create({
  model: "Qwen/Qwen3-32B",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Summarize the water cycle in one sentence." },
  ],
});

console.log(resp.choices[0].message.content);

Model ids are namespaced, like Qwen/Qwen3-32B. Discover what’s available with GET /v1/models and pass any returned id as model.

The response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "Qwen/Qwen3-32B",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "..." },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 18,
    "total_tokens": 42
  }
}

string

Unique id for this completion.

array

One entry per generated choice. Each has an index, a message (role + content, plus tool_calls when tools are used), and a finish_reason (stop, length, tool_calls, …).

object

prompt_tokens, completion_tokens, and total_tokens. This reflects the exact tokens billed to your wallet.

Parameters

Omnia forwards a fixed allow-list of parameters to the model. model is required, and everything else is optional.

Parameter	Description
`model`	The model id (required). Get valid ids from `GET /v1/models`.
`messages`	The conversation so far (required). Send the full history each request.
`temperature`	Sampling randomness (typically 0–2). Lower is more deterministic.
`top_p`	Nucleus sampling: consider only the top-probability mass.
`top_k`	Sample only from the `k` most likely tokens. Beyond the OpenAI standard set.
`n`	Number of choices to generate. Each choice adds completion tokens to `usage`.
`max_tokens`	Cap on output tokens. Bounded by the model’s context window.
`max_completion_tokens`	Alias for `max_tokens`; both are accepted.
`stop`	Up to a few stop sequences; generation halts when one is produced.
`seed`	Request more reproducible output for the same inputs.
`frequency_penalty`	Penalize tokens by how often they’ve already appeared.
`presence_penalty`	Penalize tokens that have appeared at all, to encourage new topics.
`repetition_penalty`	Alternative repetition control. Beyond the OpenAI standard set.
`logit_bias`	Map of token id → bias to nudge specific tokens up or down.
`logprobs`	When `true`, return log probabilities for output tokens.
`top_logprobs`	Number of most-likely alternatives to return per position (needs `logprobs`).
`reasoning_effort`	For reasoning models: how much the model should “think” (e.g. `low`/`medium`/`high`).
`response_format`	Force JSON or a strict schema; see Structured output.
`tools` / `tool_choice`	Function calling; see Tool calling.
`parallel_tool_calls`	Allow multiple tool calls in a single turn (boolean).
`user`	An opaque end-user identifier you can attach to a request.
`stream`	Stream tokens as they’re generated; see Streaming.
`stream_options`	Streaming controls, e.g. `{"include_usage": true}`.
`fallbacks`	Up to 2 backup models to try if the primary fails; see Fallback models. Handled by the gateway, never forwarded to the model.

Unknown parameters are silently ignored, not rejected. The gateway forwards only the fields in the table above; anything else (vendor extensions, non-standard flags, store, and so on) is stripped from the request before it reaches the model. This is a deliberate security allow-list. Your request will not error; the unrecognized field simply has no effect.

max_tokens is capped at the model’s context window. Requesting more than the model supports is rejected before any tokens are generated, so you never pay for an impossible request.

Fallback models

Pass fallbacks, a list of up to 2 backup model ids, and the gateway runs a retry-then-fallback ladder for you: the primary model is retried once on a transient failure, then each fallback is tried in order. You only ever get a 502 after every model in the chain has failed.

curl https://gateway.omnia-voice.com/v1/chat/completions \
  -H "Authorization: Bearer $OMNIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-4.5",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "fallbacks": ["Qwen/Qwen3-32B"]
  }'

resp = client.chat.completions.create(
    model="zai-org/GLM-4.5",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"fallbacks": ["Qwen/Qwen3-32B"]},
)
print(resp.model)  # the model that actually answered

const resp = await client.chat.completions.create({
  model: "zai-org/GLM-4.5",
  messages: [{ role: "user", content: "Hello!" }],
  fallbacks: ["Qwen/Qwen3-32B"],
});

console.log(resp.model); // the model that actually answered

How the ladder behaves:

Only transient failures descend the ladder: rate limits (429), timeouts, and upstream 5xx/connection failures. Deterministic client errors (a bad parameter, a prompt over the context window) fail identically on every model, so they return immediately without burning attempts.
You’re always told who answered. The response’s model field names the model that actually served, and when a fallback kicked in the response carries an X-Omnia-Fallback-From header naming the model it fell back from. The request detail in Observability shows the same.
You pay for what ran. Billing uses the model that actually answered, at that model’s own price. (The pre-flight balance check covers the priciest model in your chain, so a fallback can never overdraft your wallet.)
Streaming falls back only before the first token. Once output has started streaming, the gateway never switches models mid-response.
Every fallback id must be a valid, available model: an unknown id is rejected up front with a 400, never discovered mid-request.

fallbacks is supported on chat completions only, and not on dedicated endpoints (a dedicated endpoint is a specific deployment; there is nothing to fall back to).

Response caching

If your workspace has response caching enabled (it’s opt-in, off by default), a byte-identical chat request repeated within your retention window is served straight from the gateway’s cache: no model run, billed at 25% of the normal price. No request changes are needed; every cache-eligible response tells you what happened via a header:

X-Omnia-Cache: hit     # served from cache, billed at the discount
X-Omnia-Cache: miss    # generated fresh (and may be stored for next time)
X-Omnia-Cache: bypass  # you sent X-Omnia-Cache-Control — forced fresh

Individual requests can opt out without touching the workspace setting: send X-Omnia-Cache-Control: no-cache to force a fresh model run (the cached copy is refreshed), or no-store to also keep the response out of cache storage entirely; see per-request control. Works for streaming and non-streaming alike (one cache entry serves both modes; stream doesn’t change the cache key), including tool-call responses. Requests with n > 1 bypass the cache, and errors or dropped streams are never stored. How caching and fallbacks interact:

The fallbacks list is part of the cache key: the same messages with a different (or no) fallback chain are separate entries.
Only primary-served responses are stored. A response that came from a fallback model is never cached, so a cache hit always replays, and bills, the model you asked for.

See Response caching for what enabling it stores, the retention windows, and when it pays off.

Reasoning models

Reasoning models accept reasoning_effort to trade latency and cost against answer quality. Higher effort spends more time (and completion tokens) reasoning before answering.

resp = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=[{"role": "user", "content": "Prove there are infinitely many primes."}],
    reasoning_effort="high",
)

Multiple choices and log probabilities

Set n to return several independent completions in one call, and enable logprobs (optionally with top_logprobs) to inspect the model’s token-level confidence.

resp = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=[{"role": "user", "content": "Give me a tagline for a coffee shop."}],
    n=3,
    logprobs=True,
    top_logprobs=5,
)

for choice in resp.choices:
    print(choice.index, choice.message.content)

Multi-turn conversations

Omnia is stateless: it stores nothing between calls. To continue a conversation, resend the full message history: append the assistant’s previous reply and the new user message, then call the endpoint again.

messages = [
    {"role": "system", "content": "You are a concise assistant."},
    {"role": "user", "content": "What's the capital of France?"},
]
first = client.chat.completions.create(model="Qwen/Qwen3-32B", messages=messages)

messages.append(first.choices[0].message)            # assistant reply
messages.append({"role": "user", "content": "And its population?"})
second = client.chat.completions.create(model="Qwen/Qwen3-32B", messages=messages)

Errors and rate limits

Rate limits are enforced per model as tokens-per-minute (TPM) and requests-per-minute (RPM). Exceeding them returns HTTP 429; see Rate limits. For the full list of error codes and how to handle them, see Errors.

Get started

Capture

Observe

Grade & Judge

Evaluate

Improve

Run at scale

Platform

Chat completions

Basic request

The response

Parameters

Fallback models

Response caching

Reasoning models

Multiple choices and log probabilities

Multi-turn conversations

Errors and rate limits

​Basic request

​The response

​Parameters

​Fallback models

​Response caching

​Reasoning models

​Multiple choices and log probabilities

​Multi-turn conversations

​Errors and rate limits

Basic request

The response

Parameters

Fallback models

Response caching

Reasoning models

Multiple choices and log probabilities

Multi-turn conversations

Errors and rate limits