Local model store
The local model store keeps model weights inside your install, in its own object store. A model built from the store gets its weights from there, not from Ollama, Hugging Face or another website. This is how you add models to an install that can't, or shouldn't, download them.
Each stored model is called an artifact. A catalog entry points at it with a model reference like artifact://qwen-offline.
You need the platform admin role to see and change the store.
Open the store
Open Models and choose Local model store. The page lists every stored artifact with its files and sizes, and which runtimes can build from it. An artifact that doesn't yet have a full set of files for any runtime is marked.
Fill the store
Upload weights you already have
Under Upload, enter an Artifact name, pick the Runtime, and choose the files. Each runtime expects fixed filenames, and Orchestrate refuses any other name:
| Runtime | Files | What they are |
|---|---|---|
| Ollama | model.gguf and Modelfile | The GGUF weights, and the model's Modelfile. Get the Modelfile on a connected machine with ollama show --modelfile <model> > Modelfile. |
| vLLM | model.tar.gz | A Hugging Face snapshot directory, as huggingface-cli download --local-dir produces it. |
| Triton | model.onnx | The ONNX model. |
| Whisper | model.tar.gz | A CTranslate2 model directory, as faster-whisper downloads it. |
| Piper | voice.onnx and voice.onnx.json | The voice and its config file. |
| llama.cpp | model.gguf | One GGUF file. Merge a split model first with llama-gguf-split. |
| SGLang | model.tar.gz | A Hugging Face snapshot directory, the same as vLLM. |
Speech (hf-speech) | model.tar.gz | The speech model's files, safetensors or ONNX only, with a vois-speech.json that names the engine. See Voices and speech models. |
Reranker (infinity) | model.tar.gz | The reranker's Hugging Face snapshot, in safetensors. |
OCR (rapidocr) | det.onnx, rec.onnx and cls.onnx | The OCR pipeline's detector, recognizer and orientation classifier. |
An Ollama vision model can also carry its image projector as projector.gguf. Downloading an Ollama vision model into the store keeps it, so vision models build offline too.
An artifact name uses lowercase letters, digits and hyphens, starts and ends with a letter or digit, and is at most 63 characters.
A model.tar.gz must unpack with the files at the top level, not inside a folder. For vLLM, config.json must be at the top. For Whisper, model.bin must be at the top. The build stops with an error if it is not.
cd my-model-dir && tar -czf ../model.tar.gz .
The Modelfile is required for Ollama. Without it, an instruct model never gets its chat template and answers with unrelated text.
Download into the store
While your install can reach the internet, it can fetch a model's weights straight into the store, already in the right layout. Under Download into local store, search for the model with the same picker as Add model, check the Store as name, and choose Download into store. The weights come from:
| Runtime | Downloaded from |
|---|---|
| Ollama | The Ollama registry. The weights are checked against their published SHA-256 digest, and the Modelfile is composed from the model's template and parameters. |
| vLLM and Whisper | Hugging Face. A bare Whisper size, such as small, means the matching faster-whisper model. |
| Piper | The Piper voice repository, or a URL. |
| Triton | A URL to the .onnx file. |
Progress shows under Downloads. A download that an API restart cut off shows as failed. Start it again. Downloading under a name that already holds a complete artifact replaces it.
Gated Hugging Face models, whose licence you must accept first, need a Hugging Face token on the install. Contact us to set one.
Delete an artifact
Delete a stored artifact from its row in the list. This removes its files only. Catalog entries that point at it, and images already built from it, keep working, because a built image carries its own copy of the weights. New builds from it fail.
Build a model from the store
In Models, choose Add model. Stored models show in the picker as Local store. Pick one, and the form fills in artifact://<name> as the model reference. You can also type artifact://<name> yourself, with the runtime that matches the files.
Before it creates the entry, Orchestrate checks that the artifact has every file the runtime needs. If one is missing, it tells you which. Call the model through the Model Gateway with model set to artifact://<name>.
What still needs internet
Nothing, for the CPU runtimes. On an install from the release bundle, model builds start from the prebuilt runtime images in the bundle, with every software package already inside, and copy in the weights from the store. No base image is pulled and no package is installed from the internet, so a model whose weights are in the store builds and deploys on a network with no internet access.
The baseline speech models come prebuilt in the bundle, so voice works offline from the first install.
GPU runtimes (vLLM for GPU, Triton, SGLang) are not in the current bundle, so those models can't be built offline yet. See Requirements.
Signed model packs with ready-to-use catalog models (a CPU starter pack, a speech pack, an embeddings pack and a GPU pack) are planned, so that adding those models needs no weights of your own. voisctl can already import them; the packs are not available to download yet.
Bring models into an air-gapped install
Model weights reach an air-gapped install in one of three ways.
From a model pack
A model pack is a signed file of catalog models from us. Import it with voisctl models import-pack --pack <file> --add: it checks the signature and every file, uploads the weights into the store and adds each model, ready to deploy. See Model packs.
From a connected staging install
-
On a connected install, download the models into its store, add them from the picker, and check that they build and answer.
-
Copy them out with
voisctl:export VOIS_API_URL=https://<staging-api-host>
export VOIS_ADMIN_TOKEN=<admin access token>
./voisctl models pull --out ./models qwen2-5-1-5b whisper-base-enThe names are the artifact names the Local model store page lists. You get
./models/<name>/<file>for each one, plusmodels.json, which records the SHA-256 of every file. -
Carry the
./modelsdirectory into your network. -
At the site, push them into the store:
export VOIS_API_URL=https://<site-api-host>
export VOIS_ADMIN_TOKEN=<admin access token>
./voisctl models push ./models
./voisctl models listpushchecks every file againstmodels.jsonbefore it uploads anything.
In the release bundle
We can include stored models in your release bundle. Their SHA-256 hashes are then in the bundle's signed manifest, and voisctl verify checks them with everything else. After the install or upgrade, push them into the store:
./voisctl models push --bundle ./bundle --keys ./keys
push checks every file against the signed manifest before the first upload. Pass the extracted bundle directory to avoid unpacking the .tar again. Add --only <name> to push just one artifact.
In both cases the models then show in the site's picker as Local store.
The token is an access token for a platform admin. A personal API key isn't enough. See API authentication.
Use the API
The store has API endpoints too, for scripts. Upload one file per request, with the artifact name and filename in the path:
curl -X POST "$ORCHESTRATE_API/admin/model-artifacts/qwen-offline/model.gguf" \
-H "Authorization: Bearer $ORCHESTRATE_TOKEN" \
-F "file=@./model.gguf"
List an artifact's files with GET /admin/model-artifacts/qwen-offline, and delete it with DELETE /admin/model-artifacts/qwen-offline. See the Models API for every endpoint.