A training job on PrivateMind runs on standard open-source libraries, from NVIDIA's NCCL up to Hugging Face's TRL. PrivateMind adds the SDK that sets a job up and the platform that runs it on your org's GPUs.
The stack, from the GPUs up
- NCCL is NVIDIA's library for GPU-to-GPU communication. During training it moves gradients and weights between a job's GPUs.
- PyTorch, in NVIDIA's CUDA build, runs the model's maths on the GPUs. It also starts one training process per GPU and uses NCCL to keep them in step.
- Hugging Face Transformers loads the base model and its tokenizer, from a Hugging Face id, a folder on your volume or a registry version. TRL's trainers are built on its trainer.
- PEFT adds a LoRA adapter: a small set of new weights trained on top of the frozen base model. For QLoRA, bitsandbytes first loads the base model in 4 bits, so a large model fits on fewer GPUs.
- TRL holds the trainers for SFT, DPO, KTO and GRPO. A fine-tune's
taskpicks one. - Accelerate lets the trainer run across all of a job's GPUs. For a full fine-tune, DeepSpeed ZeRO-3 splits the model, its gradients and its optimizer state across those GPUs, so a model too large for one GPU can still train.
- vLLM generates text from a model. GRPO uses it to write the answers it scores at each step,
pm.evaluateuses it to answer your eval set, and model servers use it to serve your models and adapters. - Ray gives each job its own small cluster: a head with no GPU that runs your entrypoint, and GPU workers that train. KubeRay runs that cluster on Kubernetes, and the KAI scheduler starts the head and every worker at once, so a job never holds some of its GPUs while it waits for the rest.
- MLflow records each run: its configuration, the trainer's metrics as it trains and, for an adapter, the finished files. Each org that uses MLflow has its own instance.
What PrivateMind adds
- The
privatemindSDK.pm.finetunechecks your configuration and dataset, then submits the job.pm.evaluatescores a model on your eval set.run.publishmerges an adapter into its base model and pushes the result to your org's registry. A model server serves a published version, or a base model with your LoRA adapters on it. - The recipes. Three Python scripts drive the libraries above: one fine-tunes with TRL, one evaluates with vLLM and one merges an adapter for publishing. They come inside the SDK package, so you can read what your job runs.
- The platform. It turns a job into a Ray cluster on your org's GPUs, in your org's own namespace, and tracks it until it ends.
How a job runs
- You submit the job from the SDK, or with New run on the Training tab of the app. A fine-tune first checks its configuration and dataset where you call it, then copies the recipe and its configuration to your home volume.
- The platform checks the request against your org's GPUs, your personal compute quota and the list of training images, and records the job.
- It starts the job's Ray cluster in your org's namespace, on free GPUs your org owns or the ones you named. While the GPUs are busy, the run shows
Queued. - The Ray head runs the recipe. The recipe starts one training process per GPU, and each loads the model with Transformers and trains it with TRL.
- The trainer writes checkpoints and the result to
output_pathon your volume, and sends its metrics to your org's MLflow. The run ends asSucceeded,FailedorStopped, which theRunhandle reports. - From there,
run.publishputs the result in your org's registry, and a model server can serve it.
Versions in the current images
A fine-tune runs on one of two training images. SFT, DPO and KTO use the default image, workspace-pytorch-ngc, the same image a PyTorch GPU workspace runs. GRPO and pm.evaluate use training-rl, which adds vLLM.
| Library | workspace-pytorch-ngc 1.4.40 |
training-rl 0.1.0 |
|---|---|---|
| Base image | NVIDIA PyTorch container 26.10 | NVIDIA vLLM container 26.10 |
| CUDA | 13.4 | 13.4 |
| NCCL | 2.32.3 | 2.32.3 |
| PyTorch | 2.15.0a0+875d815, NVIDIA's build | 2.15.0a0+875d815, NVIDIA's build |
| Transformers | 5.23.0 | 5.17.0 |
| PEFT | 0.21.3 | 0.21.3 |
| bitsandbytes | 0.50.2 | 0.50.2 |
| TRL | 1.18.0 | 1.18.1 |
| Accelerate | 1.15.0 | 1.15.0 |
| DeepSpeed | 0.20.0 | 0.20.0 |
| vLLM | not included | 0.31.0 |
| Ray | 2.59.0 | 2.59.0 |
| MLflow | 3.18.0 | 3.18.0 |
privatemind SDK |
0.11.0 | 0.11.0 |
Leave image unset on pm.finetune or pm.evaluate and the job runs on the newest version of the image it needs. To pin a version, Client().list_training_images() lists every version on offer for both images, the default first: pass an entry's value as image=. In the app, the Image field of a new training run offers the same list. Submit a job has the details. Before training starts, a fine-tune's log names the version of each library in the image.
Model servers run vLLM in a version you pick from pm.model_server_catalog().versions.
Your data and weights
What a job reads and writes stays in your org:
- Your dataset is read from your volume. A reward function or grader you write is saved to your home volume and runs inside your own job, on your GPUs.
- Checkpoints and the finished adapter or model are written to
output_pathon your volume. run.publishpushes to your org's registry, which only your org can reach.- Parameters, metrics and adapter files go to your org's MLflow. The trainer reports to no other service.
A base model given as a Hugging Face id is downloaded from Hugging Face when the job starts. To start from weights you already hold, pass a path on your volume as model=, or a registry version as weights=.
Open source, no lock-in
Each library in the stack above is open source, and the SDK, recipes included, is Apache 2.0 licensed. An adapter is a standard PEFT folder: adapter_config.json, adapter_model.safetensors and the tokenizer files. Any tool that reads PEFT adapters can load it, on PrivateMind or elsewhere. A published model is a standard Hugging Face model folder with safetensors weights, and privatemind weights pull downloads any version to a folder you choose.
To train with other libraries or other versions, run your own code as a workspace job, in a workspace image or a container your org pushed to its registry.
Where next
- Compute and training overview: what a GPU allocation gives you, and personal compute quotas.
- Fine-tuning: SFT, methods, checkpoints and resources.
- Preference tuning: DPO and KTO on pairs of answers.
- Reinforcement learning: GRPO with a reward function you write in Python.
- Evaluate and publish: score a model, push it to your registry and serve it.
- Workspace jobs: one command in a workspace image or your own container.
- Model servers: serve a published model, or a base model with LoRA adapters.