Get API key

Model servers

View as Markdown

Launch, manage and register model servers over HTTP.

A model server is an OpenAI-compatible vLLM server pinned to GPUs your org owns. This API does what the Deploy a model page does in the app, for scripts and CI. The same calls are in the Python SDK, and the machine-readable spec is /openapi-model-servers.yaml.

All paths below are on https://privatemind.com/api/v1/model-servers. Once a server is registered, you call it through api.privatemind.com like any other model.

Authentication

Send a personal API key in the Authorization header, access key id and secret joined by a colon:

Text
Authorization: Bearer <ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>

Inference only, Inference and containers and management keys get 403, and so does an org application key. A read-only key can only read. Launching also needs compute access and GPU access on the account.

Service identities

For a script that should not depend on one person's key, an organization admin creates a service identity with May manage model servers ticked (Settings → Organization → Service Identities). The identity is a member of your org in its own right: it has its own compute quota and GPU access, the servers it launches show its name as the launcher, and every launch, delete, register and unregister it makes lands in your audit log. Service identities are switched on per org by PrivateMind; if the panel says they are not enabled yet, ask us. Exchange the identity's client id and secret for a token, then send the token as the bearer. The secret is read from the environment, so it never appears in curl's arguments:

cURL
export PM_CLIENT_ID=pmsvc_…
export PM_CLIENT_SECRET=pmsec_…

TOKEN=$(jq -n '{client_id: env.PM_CLIENT_ID, client_secret: env.PM_CLIENT_SECRET}' \
  | curl -s "https://privatemind.com/api/auth/service/token" \
      -H "Content-Type: application/json" -d @- \
  | jq -r .access_token)

curl -s "https://privatemind.com/api/v1/model-servers" \
  -H "Authorization: Bearer $TOKEN"

A token lasts 15 minutes (expires_in). A script that waits for a server to come up before calling gateway-register should exchange for a fresh token first. An identity without the model servers grant gets 403. A disabled identity, a rotated secret, or an org with service identities switched off gets 401 on the exchange and on every call. If an identity you want to retire still owns running servers, disable it. Deleting it leaves the servers running with no launcher shown.

See what you can run

cURL
curl -s "https://privatemind.com/api/v1/model-servers/catalog" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

body holds the catalog models, each with its slug, modelType and compatibleVllmVersions. total.supportedVersions holds the vLLM versions on offer.

cURL
curl -s "https://privatemind.com/api/v1/model-servers/allocations" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

total.hosts holds your org's GPU hosts. You can launch on a host where deployable is true. claims shows what already holds each card. On a host dedicated to your org, dedicated is true and you also get its name, allocatable_cpu and allocatable_memory_gib. On shared hardware host is an opaque ID such as host-3f9a1c2e, which you can pass back as gpuHost:

JSON
{
  "success": true,
  "total": {
    "org": "your-org",
    "hosts": [
      {
        "host": "gpu-host-1",
        "cluster": "gpu-cluster",
        "deployable": true,
        "total_gpus": 8,
        "dedicated": true,
        "allocatable_cpu": 344,
        "allocatable_memory_gib": 2011,
        "gpu_indices": [0, 1, 2, 3, 4, 5, 6, 7],
        "claims": {}
      }
    ]
  }
}
cURL
curl -s "https://privatemind.com/api/v1/model-servers/volumes" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

body holds the volumes you can see. Each one's metadata.name is the full name a launch needs, such as models-ab1cd.

Launch a server

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "targetCluster": "gpu-cluster",
    "gpuCount": 8,
    "vllmVersion": "v0.32.0",
    "weightsVolume": "models-ab1cd",
    "weightsSubPath": "my-model",
    "displayName": "my-model",
    "cpu": "64",
    "memory": "512Gi",
    "args": {"tensor-parallel-size": "8"}
  }'
Field Required Notes
targetCluster yes The cluster to run on.
gpuCount yes How many GPUs, up to 8, or MIG slices in Shared mode.
vllmVersion yes One of supportedVersions.
modelSlug one of these three A catalog model. Its compatibleVllmVersions must include the version.
weightsVolume one of these three The full name of a volume holding your own weights. A name that doesn't exist is accepted, and the server then stops at WeightsUnresolved.
weights one of these three A <name>:<version> from your org's registry, pinned to its digest at launch. A version that doesn't exist or is over 100 GiB is refused with 400, and one private to another member with 403.
weightsSubPath no The model's folder inside the volume. Defaults to the volume root.
gpuHost, gpuIndices no Name both to pin the server, with one index per GPU. Leave both out and free cards are picked for you.
gpuSharingMode no Exclusive, the default, or Shared for a MIG slice. Shared needs gpuHost and exactly one index in gpuIndices.
gpuMemoryUtilization no vLLM's --gpu-memory-utilization, as a string.
cpu no Total vCPU, not per GPU, as a string. Defaults to 4.
memory no Host memory as a string, a whole number in Mi, Gi or Ti, such as 512Gi.
sharedMemory no The size of /dev/shm, in the same units, such as 600Gi. It comes out of memory, so it can't be larger. Blank keeps the standard size. On a KV cache host it is at least 1.5 TiB, and a larger value raises it.
args no Extra vLLM flags without the leading dashes, all values as strings. This is how you set a KV cache.
displayName no The name shown in the app, and the start of the server's name.
nativeApi no true to launch with No translation: registering the server turns it on. Left out, it is off.
image no Your own image, which needs gpuHost and gpuIndices.
adapters no Up to 32 LoRA adapters to serve on the model, as a list of {"name": "<name>", "path": "<folder>"}.
volumes no Up to 8 of your volumes to mount read-write, as a list of {"volume": "<full name>", "mountPath": "/absolute/path"}, such as a cache that should survive a restart. They are fixed at launch. See Writable volumes for the rules; a launch that breaks one is refused with the reason.

The response is 201, with the server's generated name:

JSON
{
  "success": true,
  "total": {
    "namespace": "...",
    "name": "my-model-ab1cd",
    "spec": { "targetCluster": "gpu-cluster", "gpuCount": 8, "gpuHost": "gpu-host-1" }
  }
}

Read a server

cURL
curl -s "https://privatemind.com/api/v1/model-servers/my-model-ab1cd" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

total.cr is the full server object. Read its state from the condition of type Ready. A server just launched may have no status yet.

JSON
{
  "success": true,
  "total": {
    "namespace": "...",
    "name": "my-model-ab1cd",
    "cr": {
      "spec": { "gpuHost": "gpu-host-1", "gpuIndices": [0, 1, 2, 3, 4, 5, 6, 7] },
      "status": {
        "conditions": [
          { "type": "Ready", "status": "True", "reason": "Reconciled", "message": "vLLM is serving." }
        ]
      }
    }
  }
}
reason Meaning
WorkloadStarting Still starting.
Reconciled Serving, with status True.
ContainerCrashing The container crashed. message holds the end of its output.
WeightsUnresolved The weights can't be reached. message says why. A volume name that doesn't exist stays here, so delete the server and launch again.
VolumesUnresolved A writable volume can't be mounted. message says why. A Local NVMe volume on another host stays here, so delete the server and launch again.
Unschedulable It can't be placed yet. message says why.
FailedCreate The cluster refused the server. message says why.

Other reasons can appear: read message.

List servers

cURL
curl -s "https://privatemind.com/api/v1/model-servers" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

body holds one server object per server, with the same shape as total.cr, plus its API state: registered, gatewayApiModelName, gatewayAliases, gatewayNativeApi, and adapters, one {"name", "model", "registered", "aliases"} per LoRA adapter.

Register a server

A server isn't on the API until you register it. It has to be serving first: registering earlier answers 409.

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers/my-model-ab1cd/gateway-register" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"aliases": ["my-model"]}'

The model id is the server's name. aliases adds up to three names to call it by. Each is 1 to 63 letters, digits, dots, underscores or dashes, and starts and ends with a letter or digit. A name already taken answers 400. Registering again replaces the aliases, so a call with none clears them. A chat model's name then works on the chat completions API.

nativeApi, true or false, turns No translation on or off. If you leave it out, a new registration takes the setting the server was launched with, which is off unless the launch sent nativeApi: true. A registered server keeps the setting it has. A value you send also becomes the server's own setting, so a later registration keeps it. To flip it on a registered server, register again with the other value and send its aliases again in the same call:

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers/my-model-ab1cd/gateway-register" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"aliases": ["my-model"], "nativeApi": true}'
cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers/my-model-ab1cd/gateway-unregister" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

Unregistering takes the server off the API and leaves it running. It takes the server's LoRA adapters off too.

LoRA adapters

A server can serve LoRA adapters on its model, each callable as its own model. Name each adapter and the folder that holds it:

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "targetCluster": "gpu-cluster",
    "gpuCount": 1,
    "vllmVersion": "v0.32.0",
    "weightsVolume": "home-ab1cd",
    "weightsSubPath": "base/qwen3-8b",
    "memory": "40Gi",
    "adapters": [
      {"name": "statements-v1", "path": "adapters/statements-v1"},
      {"name": "notes-v2", "path": "adapters/notes-v2"}
    ]
  }'
  • name is 1 to 40 lowercase letters, digits and dashes, starting and ending with a letter or digit.
  • path is the adapter's folder, the one holding adapter_config.json. A relative path is read from weightsVolume, or from the weights version. An absolute path must sit inside one of the server's volumes, which is how you serve adapters on a catalog model.
  • Each adapter must be trained on the server's model. An adapter with a rank above 16 needs "args": {"max-lora-rank": "<rank>"}, a value from 1, 8, 16, 32, 64, 128, 256, 320 or 512.
  • Up to four adapters run in the same batch. "args": {"max-loras": "<n>"} raises that, at the cost of GPU memory.
  • args cannot carry lora-modules itself, and a server with adapters cannot be pooled.

Registering the server registers each adapter as its own model, named <server name>-<adapter name> and callable by its adapter name too. To call an adapter by other names, give them per adapter in the same call; an empty list leaves only the full name:

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers/my-model-ab1cd/gateway-register" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"aliases": ["qwen3-8b-base"], "adapters": {"statements-v1": ["statements"], "notes-v2": []}}'

The response lists each adapter with its model name and aliases. The same naming rules apply as for aliases. Registering again moves the names, and an adapter left out of the call goes back to its own name. An adapter name the server does not serve answers 400.

Delete a server

cURL
curl -s -X DELETE "https://privatemind.com/api/v1/model-servers/my-model-ab1cd" \
  -H "Authorization: Bearer $PMIND_USER_KEY"

This stops the server, releases its GPUs and takes it off the API.

JSON
{ "success": true, "total": { "namespace": "...", "name": "my-model-ab1cd", "deleted": true } }

Run your own image

You can run your own vLLM image on a host your org owns. PrivateMind switches custom images on per org, so ask your PrivateMind contact to start.

Build from a vLLM image, so that /bin/sh and vllm are there and the image starts the server from the arguments it's given:

dockerfile
FROM vllm/vllm-openai:v0.32.0
COPY my-vllm/ /opt/my-vllm/
RUN pip install --no-deps /opt/my-vllm

Log in with your API key and push under your org's path:

cURL
docker login privatemind.com -u "$ACCESS_KEY_ID" -p "$SECRET_ACCESS_KEY"
docker build -t privatemind.com/customers/your-org/vllm:my-build .
docker push privatemind.com/customers/your-org/vllm:my-build

List what you've pushed and how much of your storage it uses:

cURL
curl -s "https://privatemind.com/api/v1/model-servers/images" \
  -H "Authorization: Bearer $PMIND_USER_KEY"
JSON
{
  "success": true,
  "body": [
    {
      "image": "customers/your-org/vllm:my-build",
      "digest": "sha256:...",
      "size_bytes": 10737418240,
      "created_at": "2026-10-30T12:00:00Z"
    }
  ],
  "total": { "path": "customers/your-org", "used_bytes": 10737418240, "quota_bytes": 214748364800 }
}

Launch it by adding image, pinned to a host your org alone holds:

cURL
curl -s -X POST "https://privatemind.com/api/v1/model-servers" \
  -H "Authorization: Bearer $PMIND_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "targetCluster": "gpu-cluster",
    "gpuHost": "gpu-host-1",
    "gpuIndices": [0, 1, 2, 3],
    "gpuCount": 4,
    "vllmVersion": "v0.32.0",
    "image": "customers/your-org/vllm:my-build",
    "weightsVolume": "models-ab1cd",
    "displayName": "my-model"
  }'

vllmVersion still sets which flags args accepts. The tag is resolved to a digest at launch, so pushing that tag again doesn't change a running server. Registering, KV cache and monitoring work as they do on any model server.

Your org has a storage limit. Once it's full, a new push answers denied, and deleting a tag frees the space:

cURL
curl -s -u "$ACCESS_KEY_ID:$SECRET_ACCESS_KEY" -X DELETE \
  "https://privatemind.com/v2/customers/your-org/vllm/manifests/my-build"

Give each build its own tag. We keep a repository's 30 most recently pushed tags, plus any tag pushed in the last 30 days. An image whose tag was pushed over is removed, and a server still using it can't restart.

Errors

A refused call has "success": false and a reason in error or message. error is usually a string and can be an object with a type and message.

Status Cause
400 A bad field, such as an unsupported vllmVersion, an unknown modelSlug, gpuHost without gpuIndices, an alias already taken, or a nativeApi that isn't true or false.
401 A key that isn't valid. No key, on any call but a launch. A service identity token that has expired, or whose identity is disabled, whose secret was rotated, or whose org has service identities switched off.
402 The key's budget is spent.
403 error says which of these it is: no key on a launch; no compute or GPU access on the account; a key type this API refuses; a service identity without the model servers grant; a volume you can't use; a request over a personal limit or the org's budget on that cluster; a card the org doesn't own, or one already held beyond the shared memory cap; or a value the cluster refuses, such as an args value or a Shared request on a card without MIG.
404 No such server in your org.
409 No host with enough free GPUs and memory for an unpinned launch, or a server that isn't serving yet when you register it.
422 A malformed value, such as memory without a unit the API takes.
5xx Something failed on our side. Retry reads. Check before retrying a launch.