Skip to main content

Architecture and concepts

VoisX Orchestrate is a set of services that you install into one Kubernetes namespace with a Helm chart. This page explains what each part does, how a request moves through them, and the objects you create.

The services​

PartWhat it does
Admin consoleThe web app you sign in to. You manage models, agents, knowledge bases, connectors, evals, governance and access from here.
APIThe control-plane REST API behind the console. It stores projects, the model catalog, agents, knowledge bases, connectors and eval suites in Postgres, and it starts work on the orchestration engine.
Model GatewayThe one endpoint for every model call: chat, embeddings, speech-to-text and text-to-speech. Each call is authenticated, checked against policy and written to the audit log before it reaches a model. Chat and speech text is scanned for personal data.
OrchestrationAgent runs, model builds, model deploys, document ingestion and eval runs are durable Temporal workflows. A run that waits for a person, or for a slow model, keeps its state and resumes where it stopped.
Model servingThe deployment controller builds a serving image for each model inside your cluster and deploys it as a KServe InferenceService. Runtimes: Ollama, vLLM, Triton, Whisper and Piper.
Tool executionRuns connector tools, such as a SQL query or a REST call, for agents. It reads connector credentials from HashiCorp Vault and asks the policy engine before every call.
Knowledge ingestionStores uploaded documents in object storage (MinIO), splits them into chunks, embeds them through the Model Gateway and writes the vectors to Qdrant. It also serves knowledge base searches.
Policy engineOpen Policy Agent decides whether each model call, deploy and tool call is allowed, and which tool calls need a person to approve them.
Audit serviceAn append-only, hash-chained log of decisions and privileged actions, in its own database.
Licence serviceChecks your signed licence file locally, with no network call.
Eval workerRuns eval suites against agent versions and grades the results.
VoiceA LiveKit media server and a voice agent worker. The worker turns speech into text, gets a reply from the agent and speaks it back, all through the Model Gateway.
SIP gateway (optional)A Kamailio edge and a media bridge that connect phone calls from your carrier or PBX to agents. See Phone calls (SIP).
Code agent runtime (optional)Builds your agent code into signed images and runs each version as a Deployment in its own namespace, behind a default-deny NetworkPolicy. The agent edge gives code agents the platform's tools over MCP, takes their traces, and forwards their A2A calls. See Code agents.
Site gateway (optional)The endpoint that site agents on customer premises dial out to in hybrid mode. See Hybrid sites.
IdentityKeycloak holds users and their roles. The console and the API trust its tokens.
Data storesPostgres (records), Redis (the gateway's response cache), Qdrant (vectors) and MinIO (documents and imported model weights).

All of these run in your cluster. They stay inside it unless an operator exposes them. The parts an operator can expose are the console, the API, Keycloak's sign-in pages, the Model Gateway, LiveKit for voice and, for direct uploads, MinIO. The SIP gateway and the site gateway have their own settings. See Configuration.

How a request flows​

This is what happens when someone sends a message to an agent:

  1. The API checks the caller's Keycloak token and their role on the agent's project.
  2. The API starts an agent run on the orchestration engine and waits up to two minutes for the answer.
  3. The orchestration worker sends the conversation to the Model Gateway.
  4. The gateway scans the prompt for personal data, asks the policy engine whether the call is allowed, checks its response cache, and routes the call to the model. Every decision is written to the audit log.
  5. If the model asks to call a tool, the run checks whether the call needs human approval. If it does, the run waits until a person approves or rejects it.
  6. Tool execution asks the policy engine again, reads the connector's credentials from Vault and runs the call. The result goes back to the model.
  7. The run ends with the agent's answer and a trace of every step and tool call. You can open it in Runs & Traces.

Built for air-gapped networks​

  • External providers are off. The Model Gateway serves only models deployed in your cluster. See External providers.
  • The licence is checked offline. It is a signed file that the licence service verifies with a public key. See Licence.
  • Upgrades are offline bundles. A release is a signed file that you verify and apply with the voisctl tool. See Upgrade.
  • Model weights can be imported. You can upload weights into the platform's own object store instead of downloading them during a build. See Local model store.
  • Model builds still reach out. The in-cluster build of a new model image downloads base images and packages from public sources. See Requirements.

Objects you work with​

Organization, folder and project​

An organization is the top level. A standalone install has one organization. An install in SaaS mode hosts several, isolated from each other (see Multi-tenancy). Inside an organization you can group projects into folders, up to 10 levels deep. Agents, connectors, knowledge bases, eval suites and voice sessions belong to a project. People get access through a role on the organization, a folder or a project. See Users, roles and projects.

Model​

A model is an entry in the model catalog. It has:

  • a catalog id, such as qwen2-5-1-5b, which names its Kubernetes objects
  • a model reference, such as qwen2.5:1.5b, which is the name you send to the Model Gateway
  • a kind: chat, embedding, speech-to-text or text-to-speech
  • a runtime: Ollama, vLLM, Triton, Whisper or Piper

The catalog is shared by every project. A model is built once, then deployed. A deployment is either shared (one deployment every project can call) or isolated (private to one project). See Models and serving.

Agent​

An agent is a system prompt, a chat model and, optionally, tools and a knowledge base. A workflow is an agent with several steps, branches, parallel lanes or loops. A code agent is an agent written in code with a framework such as LangGraph or ADK. Every save or build creates a new version. Each agent has one live version, which is what runs when anyone calls the agent: through the Agent API, a hosted chat, a channel, A2A or a phone call. See Agents and How agents are exposed.

Connector and tool​

A connector links a project to an outside system, such as a PostgreSQL database, a REST API, a GraphQL API, Salesforce or Microsoft 365. Each connector gives agents one or more tools. See Tools and connectors.

Knowledge base​

A knowledge base holds documents that agents can search. It is tied to one embedding model for its whole life. See Knowledge bases.

Run and trace​

Every agent conversation turn, model build, deploy, ingestion and eval is a run. An agent run records a trace: each step, each model call and each tool call with its arguments, result and timing. See Runs and traces.

Approval​

An approval is a tool call that is waiting for a person. Tools that can change data wait for approval by default. See Human approval.

Eval suite​

An eval suite is a set of test cases for one agent. Each case sends a message and checks the reply and the tool calls. A project can require a passing eval before an agent version goes live. See Evals.

Policy, guardrail and audit event​

A policy decides what is allowed. A guardrail looks for personal data in prompts and replies. An audit event records each decision. See Policy, PII guardrails and Audit log.

API key​

A personal API key (vcpk_) lets your own code call the Model Gateway, and belongs to the person who created it. A project API key (vcpj_) lets an application call a project's agents through the Agent API. An agent key (vcag_) is one agent's identity at the Model Gateway and over A2A. See Users, roles and projects.

Was this page helpful?