Skip to main content

Your own models and containers

Besides the model picker and the local model store, there are two more ways to bring a model in:

  • Add from a URL: weights from an address you name, such as an internal mirror or object storage, downloaded into the local model store and checked against their SHA-256.
  • Bring your own container: any image that serves the OpenAI API, deployed and governed like every other model.

Add from a URL​

You need the platform admin role.

Requires allowed hosts

Adding from a URL is off until your operator lists the hosts models may be downloaded from, with modelStore.urlImport.allowedHosts: exact names, or *.example.com for subdomains. The connector sandbox's address rules apply on top. See Configuration.

  1. Open Models > Local model store and choose Add from a URL. The form lists the allowed hosts and the largest file allowed, 200 GiB by default.
  2. Pick the Runtime. The form asks for each file that runtime needs. See Upload weights you already have.
  3. For each file, enter its URL and its SHA-256 (64 hex characters).
  4. For Ollama, paste the Modelfile (the output of ollama show --modelfile <model>), or give a URL for it.
  5. Enter the Licence the model is under, such as Apache-2.0. It is shown with the model in the picker.
  6. Check Store as, the artifact name, and choose Download into store.

Progress shows under Downloads. A file whose SHA-256 doesn't match is refused, and the model can't be used. A download that stops can be Resumed from the parts already downloaded, or Retried. To replace a complete artifact stored under the same name, tick Replace.

Then build the model from artifact://<name> as from any stored model. See Build a model from the store.

Bring your own serving container​

Register an image that serves the OpenAI API. Orchestrate mirrors it into your model registry, deploys it with KServe like every other model, and routes calls to it through the Model Gateway, with the same access, budget, guardrail and audit controls. You need the platform admin role.

  1. Open Models and choose Bring your own container.
  2. Model: a Display name, an Id (lowercase letters, digits, - and ., at most 57 characters; it names the image and the deployment), the Served model name your server answers to, the Kind (chat or embedding) and the model's Licence.
  3. Container: pick a Server preset to fill in the port, health path and arguments it documents: vLLM's OpenAI server, Text Generation Inference, llama.cpp server, or another OpenAI-compatible server. Then enter the Image, with its registry host and a pinned tag or digest, the Port, the Health path (it must answer 2xx once the model is loaded) and any Arguments, one per line.
  4. Environment: variables for the container. Tick Secret for a key or token: its value goes to Vault, never to the database, and reaches the container as a Kubernetes Secret. Variables the platform sets are refused.
  5. Resources: a size. Pick a GPU size only if the image uses GPUs.
  6. Choose Register and mirror.

The image is mirrored into the model registry by a build, so clusters never pull it from its source. Deploy it from Models once the build is ready. The deployment's readiness follows the health path.

What Orchestrate does to keep a container it didn't build in check:

  • It runs with no privilege escalation, the NET_RAW capability dropped, the default seccomp profile and no service account token.
  • The deployment controller can create and replace the container's Secret but can never read a Secret.
  • The licence you gave is shown with the model.
Requires a registry the build can pull from

The mirroring build pulls the image from its source registry. On a network without internet access, your operator can map source registries to pull-through proxy projects in your own registry with modelServing.byo.proxyProjects, or you push the image into your registry first.

Was this page helpful?
On this page
Talk to our teamQuestions about licensing or sizing VoisX Orchestrate for your site? We can help.Talk to sales →