Compute and training need a dedicated GPU allocation for your organization. Without one, these features don't appear in your account or the API. Contact your account team to add GPU compute.
With a dedicated GPU allocation, your organization can train models on PrivateMind as well as run them: submit training jobs, fine-tune open models on your own data, work in GPU-backed notebooks and track runs in MLflow. It all runs inside your own infrastructure, so your code, data and model weights never leave your environment.
How training works describes the open-source libraries a job runs on and the versions in the current images.
What you get
- Training jobs: describe a job once and run it on your allocation's GPUs, from a notebook, a script or CI, with the Python SDK. A job runs on one node, with up to 8 GPUs.
- GPU workspaces: Jupyter notebooks on your GPUs, with the SDK and common ML libraries preinstalled and connected to your allocation.
- Fine-tuning in one call:
pm.finetunetakes a base model and a JSONL dataset and trains it with no training loop to write. It runs supervised fine-tuning (SFT), preference tuning with DPO or KTO, or reinforcement learning with GRPO. It trains a LoRA or QLoRA adapter or all the model's weights, saves checkpoints you can resume from, and checks the config and dataset before allocating a GPU. See Fine-tuning. - Evaluate and publish:
pm.evaluatescores a model on your own eval set, andrun.publishputs a fine-tuned model in your org's registry, where a model server can serve it. See Evaluate and publish. - Experiment tracking: if MLflow is set up for your org, jobs record their runs in a managed instance. If it isn't, turn MLflow off for the job, or the job is refused. A fine-tune that can't reach MLflow while it runs carries on untracked and says so in its log.
How it works
You submit a job with the Python SDK, naming an entrypoint command, one of the platform's training images and how many GPUs it needs. The platform schedules it onto your allocation, runs it and, with MLflow on, tracks it there. You get back a handle you can poll, wait on, stop or delete. If the job fails, the handle says why and where the full log is.
Training jobs run on the platform's own training images only. You can't submit one with an image of your own. To run your own environment on your GPUs, use a workspace job.
In a PrivateMind workspace the SDK is preinstalled and configured, so a notebook's first cell can submit a job. From a laptop or CI, authenticate with a PrivateMind API key and name your GPU cluster explicitly.
Personal compute quotas
Your workspaces, inference deployments, training runs and workspace jobs all draw from one personal allowance of host RAM and vCPU, not one per kind of workload. The Workspaces page shows how much you're using.
Host RAM is not GPU VRAM. RAM is the machine's main memory, used by your code, the operating system and CPU-side data. VRAM is on the GPU and holds model weights, activations and other GPU data. /dev/shm is shared memory in host RAM and counts toward the workload's memory request, so a larger /dev/shm leaves less RAM for everything else in that workload.
A training run's memory and vCPU total includes its Ray head plus every worker. The Ray head is the node that runs your entrypoint. For example, a worker at 128 GiB plus the 16 GiB head count as 144 GiB. A fine-tune sizes its worker from its GPUs unless you set the size: with 8 GPUs it asks for 32 vCPU and 128 GiB. A run that has finished keeps its share until its Ray cluster is removed, which happens ttl_seconds_after_finished after it ends (10 minutes unless you set it). A stopped run gives its share back straight away.
Your personal headroom is separate from your organisation's allocation and the physical host's free capacity. Personal headroom doesn't guarantee that your organisation has capacity left or that a host can take the workload, and free hardware doesn't let you go over your personal cap. A pending workload already counts against your personal RAM and vCPU, though it uses no host RAM until its pods are scheduled.
To free personal quota, stop a workspace, remove an inference deployment, stop a training run or delete a workspace job. A stopped workspace keeps its files but releases its RAM and vCPU. You can also request smaller sizes for new workloads or reduce a workspace's compute. For a higher ceiling, ask your org admin to raise your personal memory or vCPU limit. If you're within your limits and still can't launch, the constraint may be your organisation's allocation or free host capacity.
Start here
- Python SDK: how the SDK is configured and authenticated, in a workspace and outside one.
- Training jobs: submit, watch, stop and debug training jobs from any notebook, script or shell.
- Fine-tuning: SFT with LoRA, QLoRA or full-parameter training, without writing a training loop.
- Preference tuning: DPO and KTO on pairs of preferred and non-preferred answers.
- Reinforcement learning: GRPO with a reward function you write in Python.
- Evaluate and publish: score a model on your eval set, push it to your registry and serve it.
- Workspace jobs: run one command in a workspace image, once or on a schedule.