# Fine-tuning One call to fine-tune a model with SFT, DPO, KTO or GRPO, as a LoRA or QLoRA adapter or with every weight trained. The SDK validates the config and the dataset locally, then runs the training job for you. `pm.finetune(...)` runs a fine-tuning job in one call. You give it a base model, a JSONL dataset and an `output_path` for the result. The SDK checks the configuration and the dataset before it submits anything. It then submits a normal [training job](https://docs.privatemind.com/training-sdk.html) that runs a TRL recipe on your org's GPU allocation, tracked in MLflow. It returns the usual [`Run`](https://docs.privatemind.com/training-sdk.html#the-run-handle) handle, with `wait()`, `watch()`, `logs()`, `stop()` and `publish()`. The examples run as they are in a PrivateMind GPU [workspace](https://docs.privatemind.com/workspaces.html). For configuration, authentication and errors, see the [SDK overview](https://docs.privatemind.com/sdk.html). > [!NOTE] > DPO, KTO, GRPO, full-parameter training, checkpoints and registry weights need SDK 0.11.0 or later: `pip install -U "privatemind>=0.11"`. ## A first run ```python import privatemind as pm run = pm.finetune( model="Qwen/Qwen3-8B-Instruct", # HF id, or a path on a Volume dataset="sft_data.jsonl", # readable here, for the pre-flight check output_path="/workspace/adapters/support-v1", method="qlora", # 4-bit nf4 base loading experiment="support-bot", ) run.wait() print(run.phase) ``` Every argument is keyword-only. `dataset`, `output_path` and one of `model` or `weights` are required. The rest have defaults chosen for a first run, such as `task="sft"`, `method="lora"` and one GPU. [Method and hyperparameters](https://docs.privatemind.com/sdk-finetune.html#method-and-hyperparameters) lists the training defaults. ## Tasks `task` picks what the model learns from: | Task | Learns from | Dataset | Guide | |---|---|---|---| | `sft` (default) | example conversations or text to imitate | `{"messages": [...]}` or `{"text": ...}` | this page | | `dpo` | pairs of a preferred and a non-preferred answer to the same prompt | preference pairs | [Preference tuning](https://docs.privatemind.com/sdk-preference-tuning.html) | | `kto` | the same pairs, each answer taken on its own as good or bad | preference pairs | [Preference tuning](https://docs.privatemind.com/sdk-preference-tuning.html) | | `grpo` | a reward function you write, applied to answers the model generates | `{"messages": [...], ...}` | [Reinforcement learning](https://docs.privatemind.com/sdk-grpo.html) | Everything else on this page applies to every task. ## Check before you submit `pm.finetune_dry_run(...)` runs the same checks on the same configuration fields and prints the resolved configuration, without submitting anything: ```python pm.finetune_dry_run( model="Qwen/Qwen3-8B-Instruct", dataset="sft_data.jsonl", output_path="/workspace/adapters/support-v1", method="qlora", experiment="support-bot", ) ``` ``` privatemind finetune (dry run — no job submitted) model: Qwen/Qwen3-8B-Instruct task/method: sft / qlora (4-bit nf4 quantization) dataset: sft_data.jsonl (job-side: /workspace/sft_data.jsonl) (chat_jsonl, validated 50 rows) output: /workspace/adapters/support-v1 (adapter directory, written on job completion) experiment: support-bot resources: 1 worker x 1 GPUs (default training image) training: lr=0.0002 epochs=3 max_seq_len=4096 packing=True batch=4x4 (effective 16) seed=42 lora: r=16 alpha=32 dropout=0.05 targets=q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj ``` It accepts the configuration fields plus `weights`, `reward`, `gpus_per_worker`, `validate_data` and `sniff_rows`, and returns the validated `FinetuneConfig`. Cluster arguments such as `image`, `volumes` and `env` work only on `finetune`. You can also build the config yourself and pass it as `config=`: ```python from privatemind import FinetuneConfig, LoraConfig cfg = FinetuneConfig( model="Qwen/Qwen3-8B-Instruct", dataset="sft_data.jsonl", output_path="/workspace/adapters/support-v1", lora=LoraConfig(r=32, alpha=64), ) pm.finetune_dry_run(config=cfg) ``` `FinetuneConfig` and `LoraConfig` are frozen: they can't be changed once built. Both reject unknown fields: a typo raises `ValidationError` naming every problem at once. `LoraConfig` does this from SDK 0.9.1; on 0.9.0 and earlier it ignores keys it doesn't know, so a mistyped `lora=` key trains at the default. ## Datasets For SFT, two dataset formats are supported: | Format | One line looks like | |---|---| | `chat_jsonl` | `{"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}` | | `text` | `{"text": "..."}` | `pm.dataset_formats()` returns every format name: `chat_jsonl` and `text`, plus `preference` and `prompt_chosen_rejected` for DPO and KTO, which [Preference tuning](https://docs.privatemind.com/sdk-preference-tuning.html#data-format) describes. GRPO reads `chat_jsonl` rows. The file must be `.jsonl`, or `.json` with one object per line. The SDK detects the format from the first row unless you pass `dataset_format=`, and refuses a format that doesn't fit the task. Parquet is not supported yet. Before submitting, the SDK checks the first 50 rows against the format; `sniff_rows=` changes the count. Each of these problems raises `ValidationError` with the row, the reason and an example of the expected shape: - a malformed line - a row that is not an object - an empty `content` - a file that is not UTF-8 If your columns have other names, map them with `column_map`, from the expected field name to your column: ```python run = pm.finetune( model="Qwen/Qwen3-8B-Instruct", dataset="sft_data.jsonl", output_path="/workspace/adapters/support-v1", column_map={"messages": "conversation"}, ) ``` Pass `validate_data=False` to skip these checks, for a dataset on a Volume this notebook hasn't mounted. The SDK then sends the path unchanged, so give the path the job will see. To measure the model on held-out data while it trains, pass `eval_dataset=` with a second file in the same format: the job evaluates it at the end of every epoch. Or set `validation_ratio=` to hold out a share of the training file. The split uses `seed`, so every training process splits the same way. At 0, the job trains on everything. The eval figures go to MLflow next to the training loss. A conversational dataset needs a base model whose tokenizer has a chat template, in practice an instruction-tuned model. To pass options to the template, use `chat_template_kwargs`, for example `{"enable_thinking": False}` for Qwen3. ## Where paths point Write a dataset or model path under your home Volume as you would in the notebook. The SDK rewrites it to the path the job sees, which the dry run shows as `job-side:`. A local path outside the home Volume raises an error, because the job could never read it. A Hugging Face model id is passed through unchanged. `output_path` is different, because the job writes it. Give an absolute path as the job sees it: under `/workspace` for the home Volume, or under a `mount_path` you pass in `volumes=`. > [!WARNING] > If `output_path` sits outside every mount, `finetune` warns but still submits, and you lose the result. The job writes it to local disk, which is deleted when the job ends. ### A base model from your registry To start from a model in your org's [registry](https://docs.privatemind.com/registry.html), pass `weights=` in place of `model=`, as `:`: ```python run = pm.finetune( weights="credit-ft:2026-11-16", dataset="sft_data.jsonl", output_path="/workspace/adapters/credit-ft-v2", ) ``` The job mounts that version read-only at `/home/privatemind/models/`, the folder named after the model, and trains from it. You can start from a version you published from an earlier fine-tune: see [Evaluate and publish](https://docs.privatemind.com/sdk-evaluate.html). ## Method and hyperparameters `method` picks what is trained: - `lora` trains an adapter on the base model in bf16, or fp16 where the GPU has no bf16. The adapter is a small set of trained weights that sits on top of the base model. - `qlora` first loads the base model compressed to 4 bits (nf4 with double quantization), so a large model fits on fewer GPUs, then trains an adapter the same way. - `full` trains every weight of the model. The job shards the model, its gradients and its optimizer state across all its GPUs with DeepSpeed ZeRO-3. It needs far more GPU memory than an adapter, and the result is a full copy of the model. | Field | Default | Accepted | |---|---|---| | `lr` | by task and method, below | greater than 0, up to `0.1` | | `epochs` | `3.0` | greater than 0 | | `max_steps` | none | `1` or more. Stops after that many optimizer steps, in place of `epochs` | | `max_seq_len` | `4096` | `128` to `131072` | | `per_device_batch_size` | `4` | `1` to `256` | | `gradient_accumulation_steps` | `4` | `1` to `1024` | | `packing` | `True` | any boolean. SFT only | | `validation_ratio` | `0.0` | `0.0` or more, below `0.5` | | `seed` | `42` | any integer | Leave `lr` out and it depends on the task and method: | Task | LoRA and QLoRA | Full | |---|---|---| | SFT | `2e-4` | `2e-5` | | DPO, KTO | `5e-6` | `1e-6` | | GRPO | `1e-5` | `1e-6` | Configure the adapter with `lora=`, as a `LoraConfig` or a plain mapping of its fields: | Field | Default | Accepted | |---|---|---| | `r` | `16` | `1` to `256` | | `alpha` | `32` | `1` or more | | `dropout` | `0.05` | `0.0` or more, below `1.0` | | `target_modules` | the seven attention and MLP projections | any non-empty list of module names | `packing` joins short samples to fill the sequence length, which trains much faster on conversational data. It needs a FlashAttention kernel to keep packed samples from attending to each other. If the job can't get one, it stops before training and says so. Pass `packing=False` then. DPO, KTO and GRPO also take `loss_type` and `beta`, which go to the trainer: see their pages. ## Checkpoints and resume Set `checkpoint_every=` to save a checkpoint every that many training steps, in a `checkpoint-` folder under `output_path`. A checkpoint holds what the trainer needs to carry on: the weights so far, the optimizer state and the step. To carry on from a checkpoint, submit again with `resume_from=` set to its folder as the job sees it, or to `"latest"` for the newest one under `output_path`. Use the same task, method and dataset as the run that wrote it: ```python run = pm.finetune( model="Qwen/Qwen3-8B", dataset="sft_data.jsonl", output_path="/workspace/adapters/support-v2", checkpoint_every=200, resume_from="latest", ) ``` This is how you continue after a run fails or you [stop it](https://docs.privatemind.com/training-sdk.html#stop-or-delete-a-run). A run can't be restarted in place. ## Resources Fine-tuning runs on one node. `gpus_per_worker` sets its GPUs, 1 to 8, and the job starts one training process per GPU. Leave `workers` at its default of 1. Unless you set `cpu_per_worker` and `memory_per_worker`, the job sizes them from its GPUs: 4 vCPU per GPU with a minimum of 4, and 16 GiB per GPU with a minimum of 32 GiB. With 8 GPUs that is 32 vCPU and 128 GiB. Both count against your [personal compute quota](https://docs.privatemind.com/training.html#personal-compute-quotas), with the job's Ray head on top. In a workspace you can leave out `target_cluster`, `home_volume` and `image`. The first two come from the workspace, and `image` from the platform's training images. Outside a workspace, pass `target_cluster` and `home_volume`, or the call raises `ConfigError`. `env`, `ray_version`, `ttl_seconds_after_finished`, `name`, `volumes`, `gpu_placements` and `client` work as they do for [training jobs](https://docs.privatemind.com/training-sdk.html). The SDK sets up the job with environment variables whose names start with `PRIVATEMIND_FINETUNE_`, so it rejects any you pass in `env`. ## The training image The recipe runs on the platform's own training images. You can't run a fine-tune on an image of your own. SFT, DPO and KTO run on the default image, which carries `transformers`, `peft`, `trl`, `accelerate`, `datasets`, `bitsandbytes` and `deepspeed`. GRPO runs on the `training-rl` image, which adds vLLM. The SDK picks the right one, and `image=` overrides it. The job's first log lines report CUDA and the GPUs it sees. A missing package stops the run there and names it. The chat template check also runs before any training starts. ## Tracking and output MLflow tracking is on by default: - The run is named after the job. - The resolved configuration is logged as parameters. - The trainer's own metrics stream to the run during training. - For `lora` and `qlora`, the finished adapter directory is attached as an artifact under `adapter`. Pass `experiment=` to group runs, or `mlflow=False` to turn tracking off. If the job can't reach MLflow, it trains anyway without tracking and its log says so once. To read metrics, parameters and artifacts back into the notebook, see [Reading results back](https://docs.privatemind.com/training-sdk.html#reading-results-back); it needs `pip install 'privatemind[mlflow]'`. When the job finishes, `output_path` holds the result on the Volume, where it outlives the job. For `lora` and `qlora` that is the adapter: `adapter_config.json`, `adapter_model.safetensors` and the tokenizer files. For `full` it is the whole model, with its config, weights and tokenizer files. To serve the result, publish it to your org's registry with `run.publish(...)` and start a model server on it. See [Evaluate and publish](https://docs.privatemind.com/sdk-evaluate.html). ## Where next - [Preference tuning](https://docs.privatemind.com/sdk-preference-tuning.html): DPO and KTO, and the preference data formats. - [Reinforcement learning](https://docs.privatemind.com/sdk-grpo.html): GRPO with a reward function in Python. - [Evaluate and publish](https://docs.privatemind.com/sdk-evaluate.html): score the result, push it to your registry and serve it. - [Training jobs](https://docs.privatemind.com/training-sdk.html): the `Run` handle, failure reasons and logs, stopping a run, and promoting a notebook function. - [SDK overview](https://docs.privatemind.com/sdk.html): configuration, authentication, and the full error catalog.