Skip to main content

Quick start

This guide takes you from a fresh install to a model call from your own terminal. It assumes Orchestrate is already installed. If it is not, start with Install.

You need:

  • the console address and a user account from your operator
  • the developer or admin role, so you can deploy a model
  • a way to reach the Model Gateway: a hostname your operator exposed, or kubectl access to the cluster

1. Sign in​

  1. Open the console address in your browser.
  2. Choose Sign in, then enter your username and password on the sign-in page, or choose your organization's identity provider.
  3. In the picker at the top of the sidebar, choose a project. Every install has a project called Default.

2. Find a chat model​

  1. In the sidebar, open Models.
  2. Look for a model with the kind Chat whose build has finished. A new install comes with a few small Ollama chat models in the catalog and builds them when it first starts.

If no chat model has finished building, add one. See Models and serving. Model builds download from the internet. On a network with no internet access, see Requirements.

Note the model's model reference, for example qwen2.5:0.5b. That is the name you send to the Model Gateway.

3. Deploy it​

  1. Choose Deploy on the model.
  2. Under Size, keep the default.
  3. Under Who can use it, choose Shared.
  4. Choose Deploy to your cluster.

The deploy checks the image, submits it to KServe and waits for it to be ready. The status moves through Verifying integrity and Waiting for Ready to Running.

4. Try it in the Playground​

  1. Open Playground.
  2. In the target picker, choose your model under Models.
  3. Type a message and send it. The reply streams in.
  4. Select Inspect on the reply to see its timing and token counts.

5. Create an API key​

  1. Open API keys > Personal keys.
  2. Under New API key, enter a label, for example quick-start, and choose Create key.
  3. Copy the key. It starts with vcpk_.
warning

The console shows the full key only once. If you lose it, revoke it and create a new one.

6. Call the Model Gateway​

If your operator exposed the Model Gateway, use that hostname. Otherwise, forward its port to your machine. Use the namespace Orchestrate is installed in:

kubectl port-forward -n vois-orchestrate svc/model-gateway 8001:8001

Then send a chat request. Set model to the model reference from step 2:

export ORCHESTRATE_GATEWAY=http://localhost:8001
export ORCHESTRATE_API_KEY=vcpk_...

curl -X POST "$ORCHESTRATE_GATEWAY/v1/chat/completions" \
-H "Authorization: Bearer $ORCHESTRATE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen2.5:0.5b",
"messages": [{ "role": "user", "content": "Say hello in five words." }]
}'

The reply carries the answer in content, the model that served it, and token counts. This example is shortened:

{
"provider": "self_hosted",
"model": "qwen2.5:0.5b",
"content": "Hello there, nice to meet you!",
"tool_calls": null,
"usage": { "prompt_tokens": 14, "completion_tokens": 9, "total_tokens": 23 },
"cached": null,
"guardrail": {
"action": "observe",
"direction": "prompt_internal",
"findings": {}
}
}

provider: "self_hosted" tells you the call was served inside your cluster. guardrail is the PII scan report. See PII guardrails.

tip

Always name a model. A request without one goes to an external provider, and fails on an install that has none enabled.

Next steps​

Was this page helpful?
On this page
Talk to our teamQuestions about licensing or sizing VoisX Orchestrate for your site? We can help.Talk to sales →