Skip to main content

Model Gateway

The Model Gateway serves every model call. Requests follow the OpenAI API format. Every endpoint here, except /health, accepts an API key or a Keycloak access token as a bearer token.

For a guided walkthrough, see Call the Model Gateway.

Each model call is checked by the policy engine and recorded in the audit log before it reaches a model. Chat and text-to-speech text is scanned for personal data first, and a transcript is scanned on the way back. Embeddings are not scanned. A call the policy engine refuses, or cannot decide on, returns 403 with Denied by policy.

Create a chat completion​

POST/v1/chat/completions

Sends a conversation to a chat model and returns its answer. With stream set to true, the answer is streamed as server-sent events instead.

Headers​

Authorizationstringrequired
Bearer followed by an API key or a Keycloak access token.

Body​

messagesarrayrequired
The conversation, oldest first. Each message has a role (system, user, assistant or tool) and content. An assistant message that called tools has tool_calls. A tool message has the tool_call_id it answers.
modelstringoptional
The model reference to call, such as qwen2.5:1.5b. Always send it. Without it the request goes to a fallback external provider, which is off on most installs.
toolsarrayoptional
Tool definitions in the OpenAI function format.
tool_choicestringoptional
Passed to the model, for example auto or none.
streambooleanoptional
true to stream the answer. Streamed answers are scanned as they stream, and are not cached.
temperaturenumberoptional
Sampling temperature. Leave it out to use the model's default.
cachebooleanoptional
false skips the response cache, both reading and writing.

Other OpenAI fields are ignored.

Response fields​

contentstring or nulloptional
The answer. null when the model only called tools.
tool_callsarray or nulloptional
The tools the model wants to call, each with id, type and function (name and arguments as a JSON string).
providerstringoptional
self_hosted for a model in your cluster, otherwise the external provider's name.
modelstringoptional
The model that served the call.
usageobject or nulloptional
Token counts as the model reported them, such as prompt_tokens, completion_tokens and total_tokens.
cachedstring or nulloptional
exact or semantic when the answer came from the cache.
guardrailobjectoptional
The PII scan report for the prompt, with the answer's report under completion. See PII guardrails.

Streaming​

A streamed response is text/event-stream. Each data: line is an OpenAI chat.completion.chunk. The stream ends with data: [DONE]. For a model in your cluster the last chunk carries usage. The prompt's PII report is in the X-Vois-Guardrail header.

Responses​

200
The answer, or the start of the stream.
400
No route to the model, or the model's runtime could not be reached. detail names the model and the reason.
401
The key or token is missing, invalid or revoked.
403
Denied by policy.
422
A PII guardrail refused the prompt (detail.code is guardrail_refused), or the body is malformed.
429 / 5xx
The model's runtime answered with this status. Its message is passed through.

Create embeddings​

POST/v1/embeddings

Turns text into vectors with an embedding model. The model you name is always used. There is no fallback.

Body​

modelstringrequired
The model reference of an embedding model, such as all-minilm.
inputstring or arrayrequired
One string, or a list of strings.

Response fields​

dataarrayoptional
One vector per input, in the same order. Each vector is an array of numbers.
providerstringoptional
self_hosted or the external provider's name.
modelstringoptional
The model that served the call.
usageobject or nulloptional
Token counts as the model reported them.

Responses​

200
The vectors.
400
No route to the embedding model, or it could not be reached.
401
The key or token is missing, invalid or revoked.
403
Denied by policy.

Transcribe audio​

POST/v1/audio/transcriptions

Turns speech into text with a speech-to-text model. Send the request as multipart/form-data, as you would to OpenAI.

Form fields​

filefilerequired
The audio, for example a WAV file.
modelstringrequired
The model reference of a speech-to-text model, such as tiny.en.
languagestringoptional
The spoken language, such as en.

Response fields​

textstringoptional
The transcript.
providerstringoptional
self_hosted or the external provider's name.
modelstringoptional
The model that served the call.
guardrailobjectoptional
The PII report for the transcript. Transcripts are observed, never changed.

Responses​

200
The transcript.
400
The file is empty, or there is no route to the model.
401
The key or token is missing, invalid or revoked.
403
Denied by policy.
429 / 5xx
The runtime's own status, for example when it is at capacity.

Synthesize speech​

POST/v1/audio/speech

Turns text into speech with a text-to-speech model. The audio is streamed back as it is made.

Body​

modelstringrequired
The model reference of a text-to-speech model, such as en_US-lessac-medium.
inputstringrequired
The text to speak.
voicestringoptional
Passed to the runtime. A Piper image carries one voice, so this has no effect there.
response_formatstringoptional
wav (the default) or pcm. pcm is raw signed 16-bit little-endian mono audio, streamed as it is made.

Response headers​

Content-Typestringoptional
audio/wav or audio/pcm.
X-Sample-Ratestringoptional
For pcm, the sample rate in Hz.
X-Vois-Guardrailstringoptional
The PII report for the input text, as JSON.

Responses​

200
The audio.
400
The input is empty, the format is not supported, or there is no route to the model.
401
The key or token is missing, invalid or revoked.
403
Denied by policy.
422
A PII guardrail refused the input.

Read the guardrail configuration​

GET/guardrails/config

Returns the PII guardrail configuration the gateway is enforcing right now, with every direction and type filled in. If the configuration file failed to load, load_error says why and the defaults are in force.

Response fields​

detectorstringoptional
The ruleset that runs, as <id>@<version>, such as vois-bundled-ruleset@1.0.0.
detector_versionstringoptional
The ruleset version.
enabledbooleanoptional
Whether scanning is on.
sourcestringoptional
defaults or file.
load_errorstring or nulloptional
Why the configuration file could not be used, if it could not.
matrixobjectoptional
For each direction, the action for each entity type.
runs_onobjectoptional
For each kind of call (chat, transcribe, speak, embed), whether the scan runs.
checked_typesarrayoptional
The entity types being checked.
disabled_typesarrayoptional
Types switched off.
ignore_patternsarrayoptional
Patterns that are never reported.
context_suppressorsarrayoptional
Words that mark a nearby number as a business reference.
max_scan_charsintegeroptional
How many characters of each text are scanned.
unknown_entity_typesarrayoptional
Types the file names that no rule detects. They can never fire.

Check health​

GET/health

Returns {"status": "ok"} when the gateway process is up. No authentication. It does not check the models.

Was this page helpful?