Requirements
VoisX Orchestrate runs in production on your own Kubernetes cluster: the OpenShift, AKS, EKS, GKE, RKE2 or upstream Kubernetes platform your teams already operate, with several nodes, your registry, your storage and your network rules. You install it with voisctl from a signed release bundle, and every image comes from your own registry. How you receive VoisX Orchestrate explains what the bundle contains. This page lists what you need in place before you install.
For a trial or a pilot you can run everything on one server with k3s. That shape has no high availability and is not for production. See Evaluation or pilot on a single server.
Supported Kubernetes platforms
Orchestrate is one Helm chart plus a set of cluster add-ons, and runs on any conformant Kubernetes 1.28 or later on linux/amd64 nodes. voisctl checks the version and every node's architecture before it installs anything. For three platforms the bundle carries a ready-made profile that sets the right storage class, pod security and publishing for that platform.
| Platform | Versions | voisctl profile |
|---|---|---|
| Azure Kubernetes Service (AKS) | Kubernetes 1.28 or later | none needed |
| Red Hat OpenShift | 4.11 or later; 4.16 or later to serve your own certificate on Routes | openshift |
| Amazon EKS | Kubernetes 1.28 or later | eks |
| Google GKE (Standard) | Kubernetes 1.28 or later | gke |
| SUSE RKE2 / Rancher | Kubernetes 1.28 or later | none |
| Upstream Kubernetes (kubeadm and other conformant distributions) | 1.28 or later | none |
| k3s on one server (evaluation and pilots only, see Evaluation or pilot on a single server) | v1.35.9+k3s1 | single-node |
GKE Autopilot is not supported: it refuses the host-network pod that voice media needs. Use GKE Standard.
Production cluster sizing
Plan two kinds of node, and optionally a third:
| Node pool | What runs there | Starting point |
|---|---|---|
| Control plane | Kubernetes itself | Managed by your provider on AKS, EKS and GKE. On OpenShift, RKE2 and kubeadm, three control-plane nodes, sized by your distribution's own guidance, with no Orchestrate workloads scheduled on them. |
| Workers | Orchestrate, the cluster add-ons, the models you deploy, model builds and code agents | At least 3 nodes, so the three PostgreSQL instances land on different nodes and a node drain never takes the database's quorum. 8 vCPU and 32 GiB per node is a reasonable start; add nodes as you deploy models and agents. Spread them over availability zones if your platform has them. |
| GPU (optional) | GPU model serving with vLLM or SGLang | Only if you serve larger models on GPUs. See GPUs. |
What the platform itself requests, so you can check your own numbers:
- Orchestrate, with the chart's defaults: about 3.9 CPU cores and 4.5 GiB of memory in requests, with limits of about 15 cores and 9 GiB.
- High availability adds each extra replica of a service (see High availability) and two more PostgreSQL instances, each requesting 250m CPU and 512 MiB by default.
- The cluster add-ons (cert-manager, CloudNativePG, KServe, OpenBao, Harbor, ingress-nginx) request about 1.5 CPU cores and 3 GiB.
- Models come on top: each model replica requests between 0.5 and 2 cores and 1 to 4 GiB, depending on its resource profile. A model build and each code agent come on top too.
These are starting points, not a sizing recommendation for your workload. Memory headroom for models is usually what decides the size. Contact us to size a deployment.
High availability
A production install runs PostgreSQL as a replicated cluster with continuous backups, runs the stateless services more than once behind PodDisruptionBudgets, and backs every store up outside the cluster. None of it is on by default. You turn it on in your values file; Install has a complete example.
For a cluster where losing any one node must keep every service up, including Qdrant, MinIO, Redis, OpenBao and Harbor, install with --profile ha on three or more nodes with replicated storage (Longhorn from the bundle, or your own Ceph or SAN class). See Production high availability.
| Part | Production setting | What you get |
|---|---|---|
| PostgreSQL | postgresHA.mode: cnpg, postgresHA.bootstrap: initdb, postgresHA.instances: 3 (2 at the very least) | A primary and two streaming replicas on CloudNativePG. A lost node or pod fails over automatically, typically in 30 to 60 seconds. |
| No loss of acknowledged writes | postgresHA.synchronous.enabled: true, method: any, number: 1, dataDurability: required, failoverQuorum: true | Every commit waits for one standby, so the promoted replica holds every write a client was told succeeded. With 3 instances, any one can fail without stopping writes. See Synchronous replication. |
| Spread over nodes | postgresHA.podAntiAffinityType: required | The scheduler never puts two PostgreSQL instances on one node. Needs at least as many worker nodes as instances. |
| Stateless services | availability.replicas, for example { api: 2, web: 2, modelGateway: 2, toolExecution: 2, orchestrationWorker: 2, realtimeAgentWorker: 2 } | Each service with two or more replicas gets a PodDisruptionBudget, so a node drain or cluster upgrade takes them down one pod at a time. |
| Backups | backups.s3 pointing at your S3-compatible storage, outside the cluster | Continuous PostgreSQL WAL archiving and nightly base backups (point-in-time recovery, recovery point at most about 5 minutes), plus nightly copies of Qdrant, MinIO and the licence. See Backups. |
| Database alerts | postgresHA.monitoring.podMonitor and prometheusRule | Alerts for a lost instance, replication lag and failing WAL archiving. Needs the Prometheus operator. |
--profile ha)Keycloak, the orchestration engine (Temporal), LiveKit, the deployment controller, Qdrant, MinIO and Redis run as one replica each. If their node fails, Kubernetes starts them again on another node, and their data is on persistent volumes and in your backups; expect a short interruption of the affected function while that happens. The bundled OpenBao and Harbor also run as one server each. If your secrets store or model registry must itself be highly available, use your own HA Vault, OpenBao or registry instead of the bundled ones (see Vault or OpenBao and Registry).
OpenBao unsealing in production
The bundled OpenBao starts sealed after every restart of its pod, which on a multi-node cluster includes every node drain and cluster upgrade that moves it. Until it is unsealed, connector, channel, code-agent and model-provider credentials can't be read. Choose how it unseals before you install:
- Auto-unseal from your KMS or HSM (recommended for production). OpenBao asks your key service for its key at start, so nobody has to act after a restart. You pass a
sealsetting for the bundled OpenBao with--prerequisite-values openbao=<file>: atransitseal against a Vault or OpenBao you already run, a cloud KMS (AWS KMS, Azure Key Vault, Google Cloud KMS), or an HSM that the bundled OpenBao version supports.voisctlthen writes recovery keys, used only for maintenance, instead of unseal keys. - Unseal keys held by people (the default). Three of five key holders unseal it with
voisctl openbao unsealafter every restart. Watchvoisctl openbao statusin your monitoring. See After a restart: unseal OpenBao.
With --profile ha, OpenBao runs as three raft servers. A restarted server is sealed while the other two keep serving, so a single restart no longer makes credentials unreadable; voisctl openbao unseal unseals every sealed server. On an air-gapped site, auto-unseal works with a transit seal against a Vault or OpenBao your site runs outside this cluster, ideally HSM-backed. Its access token sits in a Kubernetes Secret, but it can only encrypt and decrypt with one key, you can revoke it at the unsealer, and it is useless without network access to it. Running servers keep serving if the unsealer is down; only a server that restarts waits for it. Back up the unsealer's transit key: without it the data can't be opened again. Don't keep unseal keys or a static key in a Secret on the same cluster: that is the same as having no seal. We send an example seal file with the release; contact us to set it up.
Kubernetes cluster
| Requirement | Details |
|---|---|
| Namespace | Orchestrate installs into the namespace vois-orchestrate. Model builds and model deployments use this namespace by name. |
| Kubernetes version | 1.28 or later. voisctl checks it before it installs anything. |
| Node architecture | The bundle's images are built for one platform, recorded in the bundle: linux/amd64 unless we agree otherwise with you. voisctl checks every node against it. voisctl inspect shows it. |
| Container runtime | Any runtime your cluster supports. The install tools don't need Docker. |
| Pod security | Orchestrate runs every pod under the Kubernetes restricted Pod Security Standard (podSecurity.restricted: true, which every platform profile sets). One exception: LiveKit uses the host network for voice media, which the restricted standard forbids. Label the namespace enforce: baseline with warn: restricted, or run voice elsewhere. On OpenShift, grant LiveKit the hostnetwork-v2 SCC; see Install. |
| Root-only runtime pods | Model builds and Ollama-based model servers, which the deployment controller creates at runtime, must run as root. A namespace that enforces the restricted standard refuses them. On OpenShift, grant anyuid to their two service accounts only. |
Storage
You need a default StorageClass that provisions ReadWriteOnce block volumes, or you name one with persistence.storageClassName. Each store (PostgreSQL, Qdrant, MinIO, Redis) can also use its own class. Choose before the first install: a class can't be changed later.
| Platform | Class to use |
|---|---|
| AKS | managed-csi-premium (Premium SSD) or your own CSI class. |
| EKS | gp3 from the EBS CSI driver, with WaitForFirstConsumer. Install the EBS CSI driver add-on first: without it every volume stays Pending. The eks profile names gp3. |
| GKE | standard-rwo (the gke profile's choice) or premium-rwo for SSD-backed PostgreSQL and Qdrant. |
| OpenShift | OpenShift Data Foundation ocs-storagecluster-ceph-rbd, or your platform's block class. Not CephFS. |
| RKE2, kubeadm | Your CSI block class: Longhorn, Ceph RBD, Portworx, your SAN vendor's driver, or - to bind to volumes you created beforehand. With --profile ha the bundle installs Longhorn for you. |
Network filesystems such as NFS, EFS, Filestore or CephFS are a poor fit for PostgreSQL. Use a class that allows volume expansion, so you can grow a volume later. See Configuration for the sizes.
Cluster add-ons
Orchestrate needs a few add-ons in the cluster. The release bundle carries all of them, with every image they use, and voisctl install --with-prerequisites installs them in the right order from your registry, with nothing fetched from the internet. If your platform team already runs one, voisctl detects it and leaves it alone:
| Add-on | Why Orchestrate needs it | Skip it when |
|---|---|---|
| cert-manager | KServe uses it for its webhook certificates. | Your cluster runs cert-manager already (detected), for example the cert-manager Operator on OpenShift. |
| CloudNativePG and its Barman Cloud plugin | Runs PostgreSQL with replicas, failover and backups. | Your cluster runs CloudNativePG already (detected). |
| KServe v0.20.0 | Serves the models you deploy. | Your cluster runs KServe already (detected). See KServe settings. |
| OpenBao | Stores connector, cluster, model-provider and code-agent credentials. | You run Vault or OpenBao already. See Vault or OpenBao. |
| Harbor | The registry model images are built into and served from. | Model images go to a registry you run. See Registry. |
| ingress-nginx | Publishes the console, API and sign-in on your hostnames. | You have an ingress controller (detected), or you're on OpenShift and use Routes. |
To leave one out on purpose, pass --skip-prerequisite <name>.
Not in the bundle, and needed or useful in production:
| Add-on | Needed for |
|---|---|
| A NetworkPolicy engine | Code agents (the default-deny policy around each agent) and tool execution's egress policy. Most production clusters already have one: Cilium, Calico, OpenShift's OVN-Kubernetes, or your cloud's policy option on AKS, EKS or GKE. Once it enforces policy, set agents.networkPolicy.enforced: true. |
| Prometheus operator | ServiceMonitors, PodMonitors and the database alerts. See Operations. |
| NVIDIA device plugin or GPU Operator | GPU model serving. |
KServe settings
If you bring your own KServe, install it with these values. They make KServe create plain Deployments and Services for models, and stop it creating an Ingress for each model. The bundled KServe is already set up this way. OpenShift AI also ships KServe, but with a different version and default mode; Orchestrate is run against upstream KServe v0.20.0.
kserve:
controller:
deploymentMode: RawDeployment
gateway:
disableIngressCreation: true
disableIstioVirtualHost: true
Orchestrate reaches every model through the Model Gateway, on the model's in-cluster address. Don't publish model endpoints yourself: a request that reaches a model directly skips the gateway's API-key check, cache, policy checks and audit log.
Vault or OpenBao
With the bundled OpenBao, voisctl sets up everything: it initialises and unseals OpenBao, and creates the secrets engine, the Kubernetes sign-in and every role Orchestrate uses. You keep the unseal or recovery keys safe. See OpenBao unsealing in production.
If you run your own Vault or OpenBao, for example an HA cluster your security team already operates, skip the bundled one (--skip-prerequisite openbao), set vault.address to it, and set it up like this:
- Enable a KV version 2 secrets engine at the path
secret/. - Enable the Kubernetes auth method for this cluster.
- Create two roles bound to the
vois-orchestratenamespace:
| Role | Service accounts | Needs access to |
|---|---|---|
vois-orchestrate-cluster-secrets | api, deployment-controller | secret/data/clusters/* and secret/metadata/clusters/* |
vois-orchestrate-connector-secrets | api, tool-execution | secret/data/connectors/* and secret/metadata/connectors/* |
Give each role's policy create, read and update on its secret/data/... path, and read, list and delete on its secret/metadata/... path. Code agents and channels use further roles and paths for agent sources, agent secrets, runtime keys and channel credentials. Set those up when you turn code agents on.
Without a secrets store, the rest of Orchestrate works. Connector tool calls, cluster registration, code agents and channels fail with an error that names Vault.
Registry
You bring one container registry that every node can pull from: your Harbor, JFrog Artifactory, Sonatype Nexus, Red Hat Quay, Azure Container Registry, Amazon ECR, Google Artifact Registry or a plain CNCF Distribution registry, with token or Basic authentication. voisctl pushes every image from the bundle into it, including the add-ons' images, and pins each one by digest.
- Use a project or namespace for Orchestrate, for example
vois-orchestrate. Third-party images keep their original path under it, for examplevois-orchestrate/quay.io/keycloak/keycloak, so the registry must accept multi-level repository paths. - Give
voisctlan account that can push to that project. - Nodes need to pull from it. On EKS, GKE and AKS the node identity usually does this (the node role for ECR,
roles/artifactregistry.readerfor Artifact Registry, an attached ACR). Otherwise, create a pull Secret and name it inimagePullSecrets. - The nodes' container runtime must trust the registry's certificate. If your own CA signed it, keep that CA as a PEM file: you pass it to
voisctlwith--registry-ca, so that model builds trust the registry too. - Size the registry for the bundle's images (about 14 GiB once pushed) plus every model you plan to run. Model images can be several gigabytes each.
Model images
Model images are built inside the cluster and pulled by the nodes. By default they go to the bundled Harbor, which voisctl sets up with its projects, robot accounts, pull and push Secrets and certificate trust. Your nodes must reach Harbor at an address you choose (--model-registry-url) and trust its certificate. See Install.
In production you may prefer your own registry for model images too, one that is already highly available, backed up and trusted by every node. Skip the bundled Harbor (--skip-prerequisite harbor), give --model-registry-url your registry's address and project, and create these Secrets in the vois-orchestrate namespace:
| Secret | Contents |
|---|---|
harbor-push-credentials | A kubernetes.io/dockerconfigjson Secret with an account that can push to your model project and pull the runtime images from your main registry. |
harbor-pull-credentials | A pull-only account, named in modelRegistry.imagePullSecrets and modelRegistry.credentialsSecret. |
harbor-ca-cert | Your registries' CA certificates, under the key ca.crt, named in modelRegistry.caCertSecret. --registry-ca is added to it. |
Model images carry model weights, and agent images carry your code: keep those projects private. With the bundled Harbor, voisctl creates the model project public and makes both pull Secrets, so you can make it private later without breaking running models. The code-agent project is private from the start.
Vulnerability scanning
The bundle carries the image scanner's (Trivy's) vulnerability database, dated, and voisctl imports it into a public scanner project, so Harbor can scan without the internet. voisctl turns on scan-on-push and a nightly scan for the model project. Set modelRegistry.scanner.url so the System page can show how old the database is; with the bundled Harbor, voisctl sets it. See Operations.
When your main registry isn't the bundled Harbor, Harbor's scanner doesn't find the imported database yet, so scans on push fail. Models still build, deploy and run. A fix is coming.
Hostnames, DNS and TLS
Nothing is reachable from outside the cluster until you ask for it. Plan these hostnames:
| Hostname | Needed for |
|---|---|
Console, for example orchestrate.example.com | People signing in to the console. |
| Sign-in (Keycloak) | The browser goes to Keycloak's sign-in page. It can share the console's hostname, because only the sign-in paths are published. |
API, for example orchestrate-api.example.com | The console calls the API from the browser, so the API needs its own hostname. |
| Model Gateway (optional) | Your own applications calling the Model Gateway. |
Voice (optional), for example orchestrate-voice.example.com | LiveKit signalling for voice sessions from a browser. |
Model registry, for example harbor.example.internal | The nodes pull model images from it. Use a DNS name, not an IP address: Harbor's generated certificate has no IP entry. |
Each hostname must resolve, from your users' networks, to your ingress controller's load balancer (or, on OpenShift, to the router; hosts under the cluster's *.apps wildcard need no DNS work).
For TLS you have two options:
- Your own certificate. Create a
kubernetes.io/tlsSecret and the chart uses it. One SAN or wildcard certificate from your enterprise CA can cover every hostname. This works on any network, including air-gapped ones. On OpenShift Routes it needs OpenShift 4.16 or later. - cert-manager. The chart asks a ClusterIssuer that you name for each certificate. An ACME issuer such as Let's Encrypt needs internet access. On an air-gapped network, use an issuer backed by your own CA. On OpenShift, Routes get their certificates through the cert-manager OpenShift Routes add-on.
Use HTTPS: browsers refuse the microphone on a plain HTTP page, so voice needs it. The chart's loadBalancer and nodePort modes serve plain HTTP, so in production publish through an ingress controller (exposure.mode: ingress) or OpenShift Routes (exposure.mode: route).
Network
Ingress and load balancing
| Platform | How the console and API are published |
|---|---|
| AKS, EKS, GKE | An ingress controller behind your cloud load balancer. The bundled ingress-nginx creates a Service of type LoadBalancer, or use the controller your platform team already runs (exposure.className). On EKS, the AWS Load Balancer Controller (alb) also works, with TLS on the load balancer. On GKE, ingress-nginx is simpler than GKE's own gce class, which needs extra objects for voice. |
| OpenShift | OpenShift Routes through the cluster's router (exposure.mode: route, set by the openshift profile). Skip the bundled ingress-nginx. |
| RKE2, kubeadm, on-premises | Your ingress controller (RKE2 ships ingress-nginx), behind your hardware load balancer, F5, or MetalLB. |
Ports
| Traffic | Port | From |
|---|---|---|
| Console, API and Model Gateway | 443 on your ingress controller or router | Users and applications |
| Voice signalling | 443 on your ingress controller or router | Browsers, for voice |
| Voice media | 7881/TCP and 7882/UDP on the node that runs LiveKit | Browsers, for voice |
| Model registry | The port of --model-registry-url | The nodes' container runtime |
LiveKit runs on the host network of whichever node it is scheduled on, and tells browsers to send media to that node's address. Open its ports on every node that can run LiveKit, or pin LiveKit to a few nodes with livekit.nodeSelector. On clusters whose nodes have only private addresses, browsers outside your network need a TURN server (livekit.externalTurnServers). See Configuration.
Air-gapped networks
Orchestrate is built to run with no internet access, and a bundle install is checked that way: on a machine whose every outbound connection is denied, the install, a model build, a model call and an agent run complete, and no pod tries to reach the internet.
- Every Orchestrate image, every bundled component, the cluster add-ons, the model runtime images and the baseline speech models come in the release bundle.
voisctlimports them into your registry. - A bundle install turns the offline settings on by itself: no telemetry or update checks, no remote price lists, and no starter models that only the internet could build.
- Model weights come in through the local model store. Builds use the bundled runtime images, so they need no internet.
- Licences are signed files that Orchestrate checks locally, and activating one needs no connection from your cluster. See Licence.
- Upgrades arrive as signed bundles that you verify and apply yourself. See Upgrade.
- Calls to external AI providers are off unless your operator sets
externalProviders.enabled. Leave it off on an air-gapped install. See Configuration.
Plan for these points on an air-gapped network:
- STUN servers. Unless you set
livekit.stunServers, LiveKit gives browsers a list of public STUN servers. Point it at a STUN server on your network if you don't want browsers to try to reach them. - Identity providers. Single sign-on works with an identity provider inside your network. For SAML you can paste the provider's metadata instead of giving a URL.
Signed model packs (CPU starter, speech, embeddings and GPU) will let you add catalog models with no internet and no weights of your own. They are not available yet.
Backups
Production backups go to S3-compatible object storage outside the cluster: AWS S3, on-premises MinIO, Ceph RGW, NetApp StorageGRID, Dell ECS or any other S3 endpoint. Before you install, have:
- a bucket for Orchestrate's backups, with versioning or object lock if your policy asks for it;
- an access key pair that can write to it, which you put in a Kubernetes Secret;
- the endpoint's CA certificate, if your own CA signed it.
PostgreSQL archives its write-ahead log continuously and takes a base backup every night; Qdrant, MinIO, Redis and the licence are copied nightly. See Scheduled backups to S3. Back up OpenBao (or your own Vault) and your registry with your own tools as well, and keep the OpenBao keys apart from its data.
GPUs
You don't need GPUs. Most model kinds run on CPU, the baseline speech models are sized for CPU, and vLLM models can be built for CPU too.
The current release bundle carries the CPU model runtimes only. GPU runtimes (vLLM for GPU, Triton, SGLang) are not in it, so a GPU model can't be built offline from it. Contact us if you need GPU serving on an air-gapped site.
To serve larger models, vLLM and SGLang models can be deployed on GPU sizes, or on a custom size with up to 8 GPUs. For that, add a GPU node pool with the NVIDIA device plugin or GPU Operator, which voisctl doesn't install. Taint the pool so only models land there, and set modelServing.gpu.nodeSelector and modelServing.gpu.tolerations if your GPU nodes are labelled or tainted differently from the usual nvidia.com/gpu taint. GPU Feature Discovery's node labels let the console offer each GPU type the cluster has.
GPUs can also be shared between models, by time-slicing or with MIG slices on A100 and H100-class cards. That is configured in the NVIDIA device plugin. See Sizes and GPUs.
Admin workstation
You run the install and upgrade from a Linux machine that can reach your registry and your cluster's API server, such as a bastion host inside your network.
| Tool | Version |
|---|---|
kubectl (or oc on OpenShift) | A version that matches your cluster. |
| Helm | 3.14 or later. |
| Python | Python 3.9 or later, with the cryptography package. The code-agent commands also need PyYAML. |
voisctl | In the release bundle, under tools/. |
The account you install with needs cluster-admin rights for the install, because the add-ons create CustomResourceDefinitions, webhooks and their own namespaces. Day-to-day operation needs only the vois-orchestrate namespace.
Leave room on the workstation for the bundle and one extracted copy. Keep a place off the cluster, such as your secrets safe, ready for the OpenBao keys that the install writes.