Model Gateway
The Model Gateway serves every model call. Requests follow the OpenAI API format. Every endpoint here, except /health, accepts an API key or a Keycloak access token as a bearer token.
For a guided walkthrough, see Call the Model Gateway.
Each model call is checked by the policy engine and recorded in the audit log before it reaches a model. Chat and text-to-speech text is scanned for personal data first, and a transcript is scanned on the way back. Embeddings are not scanned. A call the policy engine refuses, or cannot decide on, returns 403 with Denied by policy.
Create a chat completion
/v1/chat/completionsSends a conversation to a chat model and returns its answer. With stream set to true, the answer is streamed as server-sent events instead.
Headers
AuthorizationstringrequiredBearer followed by an API key or a Keycloak access token.Body
messagesarrayrequiredrole (system, user, assistant or tool) and content. An assistant message that called tools has tool_calls. A tool message has the tool_call_id it answers.modelstringoptionalqwen2.5:1.5b. Always send it. Without it the request goes to a fallback external provider, which is off on most installs.toolsarrayoptionaltool_choicestringoptionalauto or none.streambooleanoptionaltrue to stream the answer. Streamed answers are scanned as they stream, and are not cached.temperaturenumberoptionalcachebooleanoptionalfalse skips the response cache, both reading and writing.Other OpenAI fields are ignored.
Response fields
contentstring or nulloptionalnull when the model only called tools.tool_callsarray or nulloptionalid, type and function (name and arguments as a JSON string).providerstringoptionalself_hosted for a model in your cluster, otherwise the external provider's name.modelstringoptionalusageobject or nulloptionalprompt_tokens, completion_tokens and total_tokens.cachedstring or nulloptionalexact or semantic when the answer came from the cache.guardrailobjectoptionalcompletion. See PII guardrails.Streaming
A streamed response is text/event-stream. Each data: line is an OpenAI chat.completion.chunk. The stream ends with data: [DONE]. For a model in your cluster the last chunk carries usage. The prompt's PII report is in the X-Vois-Guardrail header.
Responses
detail names the model and the reason.detail.code is guardrail_refused), or the body is malformed.Create embeddings
/v1/embeddingsTurns text into vectors with an embedding model. The model you name is always used. There is no fallback.
Body
modelstringrequiredall-minilm.inputstring or arrayrequiredResponse fields
dataarrayoptionalproviderstringoptionalself_hosted or the external provider's name.modelstringoptionalusageobject or nulloptionalResponses
Transcribe audio
/v1/audio/transcriptionsTurns speech into text with a speech-to-text model. Send the request as multipart/form-data, as you would to OpenAI.
Form fields
filefilerequiredmodelstringrequiredtiny.en.languagestringoptionalen.Response fields
textstringoptionalproviderstringoptionalself_hosted or the external provider's name.modelstringoptionalguardrailobjectoptionalResponses
Synthesize speech
/v1/audio/speechTurns text into speech with a text-to-speech model. The audio is streamed back as it is made.
Body
modelstringrequireden_US-lessac-medium.inputstringrequiredvoicestringoptionalresponse_formatstringoptionalwav (the default) or pcm. pcm is raw signed 16-bit little-endian mono audio, streamed as it is made.Response headers
Content-Typestringoptionalaudio/wav or audio/pcm.X-Sample-Ratestringoptionalpcm, the sample rate in Hz.X-Vois-GuardrailstringoptionalResponses
Read the guardrail configuration
/guardrails/configReturns the PII guardrail configuration the gateway is enforcing right now, with every direction and type filled in. If the configuration file failed to load, load_error says why and the defaults are in force.
Response fields
detectorstringoptional<id>@<version>, such as vois-bundled-ruleset@1.0.0.detector_versionstringoptionalenabledbooleanoptionalsourcestringoptionaldefaults or file.load_errorstring or nulloptionalmatrixobjectoptionalruns_onobjectoptionalchat, transcribe, speak, embed), whether the scan runs.checked_typesarrayoptionaldisabled_typesarrayoptionalignore_patternsarrayoptionalcontext_suppressorsarrayoptionalmax_scan_charsintegeroptionalunknown_entity_typesarrayoptionalCheck health
/healthReturns {"status": "ok"} when the gateway process is up. No authentication. It does not check the models.