Skip to main content

Evaluation or pilot on a single server

For an evaluation or a pilot you can run the whole of VoisX Orchestrate, with its cluster add-ons, on one Linux server running k3s, with local storage, no load balancer, no GPU and no internet access. It installs from the same signed bundle as a production install, with voisctl install --profile single-node. This is also the shape we use for our own end-to-end air-gapped release test.

Not for production

A single server has no high availability. It is a single point of failure: if the server, its disk or k3s fails, every agent, model and voice session stops, and PostgreSQL runs as one instance. Capacity is limited to what one server holds, which leaves room for only a few models and code agents. Backups go to the same disk by default.

For production, install on your own multi-node Kubernetes cluster. See Requirements and Install.

The server​

MinimumNotes
CPU and memory16 vCPU, 64 GiBEverything the profile installs, plus room for a speech model, a chat model, an embedding model, one model build and two code agents, requests about 12.7 vCPU and 21.6 GiB. CPU is the tight side; memory is the headroom models need.
Disk400 GiB, 512 GiB recommendedOne fast volume (SSD) for the bundle, your registry's copy of every image, the container runtime's unpacked copy, model images, PostgreSQL, MinIO (which also holds the PostgreSQL backups) and Qdrant.
Operating systemA systemd Linux distribution that k3s supportsUbuntu 24.04 LTS is what we test.
Architecturelinux/amd64The bundle's platform.
k3sv1.35.9+k3s1Installed offline from k3s's own release files, below.
ToolsPython 3.9 or later with cryptography and PyYAML, Helm 3.14 or laterUbuntu 24.04 has Python with both packages out of the box. k3s ships kubectl as k3s kubectl.

Our reference test server is 16 vCPU, 64 GiB of memory and one 512 GiB SSD, running Ubuntu 24.04 LTS with every outbound connection denied. On it, the install from the bundle takes about 30 minutes, most of it pushing images into the registry.

How the requests add up​

The single-node profile sets requests on every pod, so the scheduler can prove the whole install fits:

PartCPUMemory
Orchestrate (including one PostgreSQL instance, the agent edge and the package index)5.06.5 GiB
Cluster add-ons (cert-manager, CloudNativePG, KServe, OpenBao, Harbor, ingress-nginx)1.53.0 GiB
k3s itself1.22.1 GiB
Reserve: speech-to-text and text-to-speech, one CPU chat model, one embedding model, one model build, two code agents5.010.0 GiB
Total requests12.721.6 GiB

That leaves about 1.7 vCPU for anything beyond the reserve, such as a second chat model or a larger speech model. A bigger server is the way to grow a pilot; a production install moves to a cluster.

1. Install k3s offline​

k3s installs with no internet from three files of its own release: the k3s binary, k3s-airgap-images-amd64.tar.zst and install.sh at the same version. We test with v1.35.9+k3s1. Download them on a connected machine, check them against k3s's published checksums, and copy them across with the bundle.

  1. Before installing k3s, create /etc/rancher/k3s/config.yaml. The bundled ingress controller takes the server's ports 80 and 443, which k3s's own Traefik would otherwise hold:

    /etc/rancher/k3s/config.yaml
    disable:
    - traefik
    write-kubeconfig-mode: "0600"

    Keep k3s's NetworkPolicy controller on: don't pass --disable-network-policy. Code agents need it.

  2. Install k3s from the files:

    sudo install -m 0755 k3s /usr/local/bin/k3s
    sudo mkdir -p /var/lib/rancher/k3s/agent/images
    sudo cp k3s-airgap-images-amd64.tar.zst /var/lib/rancher/k3s/agent/images/
    sudo INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_VERSION=v1.35.9+k3s1 sh ./install.sh
    sudo k3s kubectl get nodes,storageclass # one Ready node; local-path (default)

2. A registry​

If you have no registry the server can use, run one on it, for example the CNCF Distribution registry with TLS from your own CA and a password, with its image imported from a tar file the same way as k3s's airgap images. Tell k3s's container runtime about it, including the credentials, in /etc/rancher/k3s/registries.yaml:

/etc/rancher/k3s/registries.yaml
configs:
"registry.example.internal:5000":
auth:
username: <user>
password: <password>
tls:
ca_file: /etc/ssl/site-ca.crt

Then restart k3s (sudo systemctl restart k3s). Also add your CA to the server's own trust store, so voisctl and Helm trust the registry.

3. Names and a certificate​

Make the console, API, sign-in and voice hostnames, and a name for Harbor, resolve to the server, both for your users and on the server itself. Use a DNS name for Harbor, not an IP address: its generated certificate has no IP entry.

Create the namespace and one TLS Secret, named vois-orchestrate-tls, that covers the console, API, sign-in and voice hostnames. Your own CA is fine:

sudo k3s kubectl create namespace vois-orchestrate
sudo k3s kubectl -n vois-orchestrate create secret tls vois-orchestrate-tls --cert=tls.crt --key=tls.key

4. Install​

Verify the bundle first, as in Install: verify the bundle. Then create a short values file. The profile already publishes the console, API and sign-in through the bundled ingress controller with the vois-orchestrate-tls certificate:

site.yaml
image:
registry: registry.example.internal:5000
exposure:
components:
web: { host: orchestrate.example.internal }
api: { host: orchestrate-api.example.internal }
keycloak: { host: orchestrate.example.internal } # may share the console's hostname
livekit:
publicHost: orchestrate-voice.example.internal
agents:
enabled: true # code agents, built from the bundled package index
wheelhouse:
enabled: true

Rehearse with --dry-run, then run the same command without it:

./voisctl install --bundle ./bundle \
--registry registry.example.internal:5000 \
--registry-user <push-account> --registry-password <password> \
--registry-ca ./site-ca.crt \
--kube-context default \
--with-prerequisites --profile single-node \
--model-registry-url harbor.example.internal:30003 \
--openbao-keys ./openbao-init.json \
-f site.yaml \
--dry-run

On k3s the kubeconfig context is usually default. With the single-node profile, Harbor listens on the server's port 30003. Everything else, including what voisctl sets up, the OpenBao keys and what to do if the install fails, is as described in Install.

What the single-node profile changes​

--profile single-node applies the chart's own one-server settings and each add-on's, from the signed bundle, under your -f files:

single-nodeDefault
Storagelocal-path for every volume; MinIO 50 GiBThe cluster's default class; MinIO 10 GiB
PostgreSQLCloudNativePG, one instance, no failoverAs configured; production uses three
BackupsPostgreSQL base backups and WAL to the install's own MinIO, kept 14 days. It is the same disk as the data: point backups at another store for real recovery. See Back up and restore.None
ReplicasOne of everything, no PodDisruptionBudgetsAs configured
GPUNone: vLLM models build for CPUCUDA vLLM
Code agentsNetworkPolicy declared enforced (k3s's controller); one build at a timeTwo builds at a time
Pod securityRestrictedOff
Publishingingress-nginx on the server's ports 80 and 443, TLS Secret vois-orchestrate-tls; console, API and sign-in onNothing published
HarborNodePort 30003LoadBalancer
RequestsOn every pod, so the whole install fits on the serverSome components set none

No Service of type LoadBalancer is created.

Keeping k3s's Traefik​

If the server keeps Traefik, skip the bundled ingress controller and use Traefik's class:

./voisctl install ... --profile single-node --skip-prerequisite ingress-nginx -f site.yaml

with exposure.className: traefik in site.yaml.

5. Let k3s trust Harbor​

Copy the ./harbor-ca.crt that voisctl wrote to the server, add it to /etc/rancher/k3s/registries.yaml, then restart k3s:

/etc/rancher/k3s/registries.yaml
configs:
"harbor.example.internal:30003":
tls:
ca_file: /etc/rancher/k3s/harbor-ca.crt

Repeat this after an upgrade that moves Harbor to a new version, because Harbor's certificate changes.

6. Open the voice ports​

LiveKit runs on the server's host network. Open TCP 7881 and UDP 7882 to the browsers that will use voice.

After a reboot​

After every reboot of the server, OpenBao starts sealed. Unseal it with the keys file from the install: see After a restart: unseal OpenBao. Then sign in and check the install as in Install.

Moving from a pilot to production​

A production install is a new install on your multi-node cluster, without the single-node profile and with the high availability settings. To carry a pilot's data across, back it up to S3-compatible storage outside the server and restore it into the new install. Contact us to plan the move.

Was this page helpful?