Get API key

Fine-tuning

View as Markdown

One call to fine-tune a model with SFT, DPO, KTO or GRPO, as a LoRA or QLoRA adapter or with every weight trained. The SDK validates the config and the dataset locally, then runs the training job for you.

pm.finetune(...) runs a fine-tuning job in one call. You give it a base model, a JSONL dataset and an output_path for the result. The SDK checks the configuration and the dataset before it submits anything. It then submits a normal training job that runs a TRL recipe on your org's GPU allocation, tracked in MLflow. It returns the usual Run handle, with wait(), watch(), logs(), stop() and publish().

The examples run as they are in a PrivateMind GPU workspace. For configuration, authentication and errors, see the SDK overview.

Note

DPO, KTO, GRPO, full-parameter training, checkpoints and registry weights need SDK 0.11.0 or later: pip install -U "privatemind>=0.11".

A first run

Python
import privatemind as pm

run = pm.finetune(
    model="Qwen/Qwen3-8B-Instruct",          # HF id, or a path on a Volume
    dataset="sft_data.jsonl",                # readable here, for the pre-flight check
    output_path="/workspace/adapters/support-v1",
    method="qlora",                          # 4-bit nf4 base loading
    experiment="support-bot",
)

run.wait()
print(run.phase)

Every argument is keyword-only. dataset, output_path and one of model or weights are required. The rest have defaults chosen for a first run, such as task="sft", method="lora" and one GPU. Method and hyperparameters lists the training defaults.

Tasks

task picks what the model learns from:

Task Learns from Dataset Guide
sft (default) example conversations or text to imitate {"messages": [...]} or {"text": ...} this page
dpo pairs of a preferred and a non-preferred answer to the same prompt preference pairs Preference tuning
kto the same pairs, each answer taken on its own as good or bad preference pairs Preference tuning
grpo a reward function you write, applied to answers the model generates {"messages": [...], ...} Reinforcement learning

Everything else on this page applies to every task.

Check before you submit

pm.finetune_dry_run(...) runs the same checks on the same configuration fields and prints the resolved configuration, without submitting anything:

Python
pm.finetune_dry_run(
    model="Qwen/Qwen3-8B-Instruct",
    dataset="sft_data.jsonl",
    output_path="/workspace/adapters/support-v1",
    method="qlora",
    experiment="support-bot",
)
Text
privatemind finetune (dry run — no job submitted)
  model:        Qwen/Qwen3-8B-Instruct
  task/method:  sft / qlora (4-bit nf4 quantization)
  dataset:      sft_data.jsonl (job-side: /workspace/sft_data.jsonl) (chat_jsonl, validated 50 rows)
  output:       /workspace/adapters/support-v1 (adapter directory, written on job completion)
  experiment:   support-bot
  resources:    1 worker x 1 GPUs (default training image)
  training:     lr=0.0002 epochs=3 max_seq_len=4096 packing=True batch=4x4 (effective 16) seed=42
  lora:         r=16 alpha=32 dropout=0.05 targets=q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj

It accepts the configuration fields plus weights, reward, gpus_per_worker, validate_data and sniff_rows, and returns the validated FinetuneConfig. Cluster arguments such as image, volumes and env work only on finetune. You can also build the config yourself and pass it as config=:

Python
from privatemind import FinetuneConfig, LoraConfig

cfg = FinetuneConfig(
    model="Qwen/Qwen3-8B-Instruct",
    dataset="sft_data.jsonl",
    output_path="/workspace/adapters/support-v1",
    lora=LoraConfig(r=32, alpha=64),
)
pm.finetune_dry_run(config=cfg)

FinetuneConfig and LoraConfig are frozen: they can't be changed once built. Both reject unknown fields: a typo raises ValidationError naming every problem at once. LoraConfig does this from SDK 0.9.1; on 0.9.0 and earlier it ignores keys it doesn't know, so a mistyped lora= key trains at the default.

Datasets

For SFT, two dataset formats are supported:

Format One line looks like
chat_jsonl {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
text {"text": "..."}

pm.dataset_formats() returns every format name: chat_jsonl and text, plus preference and prompt_chosen_rejected for DPO and KTO, which Preference tuning describes. GRPO reads chat_jsonl rows.

The file must be .jsonl, or .json with one object per line. The SDK detects the format from the first row unless you pass dataset_format=, and refuses a format that doesn't fit the task. Parquet is not supported yet.

Before submitting, the SDK checks the first 50 rows against the format; sniff_rows= changes the count. Each of these problems raises ValidationError with the row, the reason and an example of the expected shape:

  • a malformed line
  • a row that is not an object
  • an empty content
  • a file that is not UTF-8

If your columns have other names, map them with column_map, from the expected field name to your column:

Python
run = pm.finetune(
    model="Qwen/Qwen3-8B-Instruct",
    dataset="sft_data.jsonl",
    output_path="/workspace/adapters/support-v1",
    column_map={"messages": "conversation"},
)

Pass validate_data=False to skip these checks, for a dataset on a Volume this notebook hasn't mounted. The SDK then sends the path unchanged, so give the path the job will see.

To measure the model on held-out data while it trains, pass eval_dataset= with a second file in the same format: the job evaluates it at the end of every epoch. Or set validation_ratio= to hold out a share of the training file. The split uses seed, so every training process splits the same way. At 0, the job trains on everything. The eval figures go to MLflow next to the training loss.

A conversational dataset needs a base model whose tokenizer has a chat template, in practice an instruction-tuned model. To pass options to the template, use chat_template_kwargs, for example {"enable_thinking": False} for Qwen3.

Where paths point

Write a dataset or model path under your home Volume as you would in the notebook. The SDK rewrites it to the path the job sees, which the dry run shows as job-side:. A local path outside the home Volume raises an error, because the job could never read it. A Hugging Face model id is passed through unchanged.

output_path is different, because the job writes it. Give an absolute path as the job sees it: under /workspace for the home Volume, or under a mount_path you pass in volumes=.

Warning

If output_path sits outside every mount, finetune warns but still submits, and you lose the result. The job writes it to local disk, which is deleted when the job ends.

A base model from your registry

To start from a model in your org's registry, pass weights= in place of model=, as <name>:<version>:

Python
run = pm.finetune(
    weights="credit-ft:2026-11-16",
    dataset="sft_data.jsonl",
    output_path="/workspace/adapters/credit-ft-v2",
)

The job mounts that version read-only at /home/privatemind/models/<folder>, the folder named after the model, and trains from it. You can start from a version you published from an earlier fine-tune: see Evaluate and publish.

Method and hyperparameters

method picks what is trained:

  • lora trains an adapter on the base model in bf16, or fp16 where the GPU has no bf16. The adapter is a small set of trained weights that sits on top of the base model.
  • qlora first loads the base model compressed to 4 bits (nf4 with double quantization), so a large model fits on fewer GPUs, then trains an adapter the same way.
  • full trains every weight of the model. The job shards the model, its gradients and its optimizer state across all its GPUs with DeepSpeed ZeRO-3. It needs far more GPU memory than an adapter, and the result is a full copy of the model.
Field Default Accepted
lr by task and method, below greater than 0, up to 0.1
epochs 3.0 greater than 0
max_steps none 1 or more. Stops after that many optimizer steps, in place of epochs
max_seq_len 4096 128 to 131072
per_device_batch_size 4 1 to 256
gradient_accumulation_steps 4 1 to 1024
packing True any boolean. SFT only
validation_ratio 0.0 0.0 or more, below 0.5
seed 42 any integer

Leave lr out and it depends on the task and method:

Task LoRA and QLoRA Full
SFT 2e-4 2e-5
DPO, KTO 5e-6 1e-6
GRPO 1e-5 1e-6

Configure the adapter with lora=, as a LoraConfig or a plain mapping of its fields:

Field Default Accepted
r 16 1 to 256
alpha 32 1 or more
dropout 0.05 0.0 or more, below 1.0
target_modules the seven attention and MLP projections any non-empty list of module names

packing joins short samples to fill the sequence length, which trains much faster on conversational data. It needs a FlashAttention kernel to keep packed samples from attending to each other. If the job can't get one, it stops before training and says so. Pass packing=False then.

DPO, KTO and GRPO also take loss_type and beta, which go to the trainer: see their pages.

Checkpoints and resume

Set checkpoint_every= to save a checkpoint every that many training steps, in a checkpoint-<step> folder under output_path. A checkpoint holds what the trainer needs to carry on: the weights so far, the optimizer state and the step.

To carry on from a checkpoint, submit again with resume_from= set to its folder as the job sees it, or to "latest" for the newest one under output_path. Use the same task, method and dataset as the run that wrote it:

Python
run = pm.finetune(
    model="Qwen/Qwen3-8B",
    dataset="sft_data.jsonl",
    output_path="/workspace/adapters/support-v2",
    checkpoint_every=200,
    resume_from="latest",
)

This is how you continue after a run fails or you stop it. A run can't be restarted in place.

Resources

Fine-tuning runs on one node. gpus_per_worker sets its GPUs, 1 to 8, and the job starts one training process per GPU. Leave workers at its default of 1.

Unless you set cpu_per_worker and memory_per_worker, the job sizes them from its GPUs: 4 vCPU per GPU with a minimum of 4, and 16 GiB per GPU with a minimum of 32 GiB. With 8 GPUs that is 32 vCPU and 128 GiB. Both count against your personal compute quota, with the job's Ray head on top.

In a workspace you can leave out target_cluster, home_volume and image. The first two come from the workspace, and image from the platform's training images. Outside a workspace, pass target_cluster and home_volume, or the call raises ConfigError. env, ray_version, ttl_seconds_after_finished, name, volumes, gpu_placements and client work as they do for training jobs. The SDK sets up the job with environment variables whose names start with PRIVATEMIND_FINETUNE_, so it rejects any you pass in env.

The training image

The recipe runs on the platform's own training images. You can't run a fine-tune on an image of your own. SFT, DPO and KTO run on the default image, which carries transformers, peft, trl, accelerate, datasets, bitsandbytes and deepspeed. GRPO runs on the training-rl image, which adds vLLM. The SDK picks the right one, and image= overrides it.

The job's first log lines report CUDA and the GPUs it sees. A missing package stops the run there and names it. The chat template check also runs before any training starts.

Tracking and output

MLflow tracking is on by default:

  • The run is named after the job.
  • The resolved configuration is logged as parameters.
  • The trainer's own metrics stream to the run during training.
  • For lora and qlora, the finished adapter directory is attached as an artifact under adapter.

Pass experiment= to group runs, or mlflow=False to turn tracking off. If the job can't reach MLflow, it trains anyway without tracking and its log says so once. To read metrics, parameters and artifacts back into the notebook, see Reading results back; it needs pip install 'privatemind[mlflow]'.

When the job finishes, output_path holds the result on the Volume, where it outlives the job. For lora and qlora that is the adapter: adapter_config.json, adapter_model.safetensors and the tokenizer files. For full it is the whole model, with its config, weights and tokenizer files.

To serve the result, publish it to your org's registry with run.publish(...) and start a model server on it. See Evaluate and publish.

Where next