# Batch Run bulk chat-completion work from Python: build the JSONL input, upload it, create a batch, wait, and iterate the results at half the token rate. The `privatemind` SDK runs bulk chat-completion work through the [Batch API](https://docs.privatemind.com/batch.html), billed at 50% of the normal token rate. The [Batch API page](https://docs.privatemind.com/batch.html) has pricing and the server-side rules. The SDK gives you six functions and a `Batch` object that waits and iterates the results for you. The steps are the same as over HTTP: 1. Upload a JSONL file of chat-completion requests. 2. Create a batch from it. 3. Wait for a terminal status. 4. Read the output. The examples run as they are in a PrivateMind workspace, where `Client()` reads its token and base URL from the environment. For configuration, authentication and errors, see the [SDK overview](https://docs.privatemind.com/sdk.html). ## The six calls Each call maps to one HTTP endpoint. They are methods on `Client` and also module-level imports (`from privatemind import create_batch`). The module-level functions share one client per process, created on first use. Pass `client=` to any of them to use a `Client` of your own. | Call | Endpoint | |---|---| | `create_file(file=, purpose="batch")` | `POST /v1/files` | | `create_batch(input_file_id=, endpoint=, completion_window=, metadata=)` | `POST /v1/batches` | | `get_batch(batch_id)` | `GET /v1/batches/{id}` | | `list_batches(limit=, after=)` | `GET /v1/batches` | | `cancel_batch(batch_id)` | `POST /v1/batches/{id}/cancel` | | `get_file_content(file_id)` | `GET /v1/files/{id}/content` | These arguments are checked in the SDK and raise `ValidationError` before any request is sent: - `purpose` must be `"batch"` (`create_file`) - `endpoint` must be `"/v1/chat/completions"` and `completion_window` must be `"24h"` (`create_batch`) - `limit` must be between 1 and 100 (`list_batches`) ## Build the input file The input is JSONL: one JSON object per line, one line per request, in the shape the [Batch API](https://docs.privatemind.com/batch.html) defines (`custom_id`, `method`, `url`, `body`). A loop of `json.dumps` calls builds it: ```python import json docs = { "doc-1": "First quarterly report text ...", "doc-2": "Second quarterly report text ...", "doc-3": "Third quarterly report text ...", } with open("requests.jsonl", "w") as fh: for custom_id, text in docs.items(): fh.write(json.dumps({ "custom_id": custom_id, "method": "POST", "url": "/v1/chat/completions", "body": { "model": "default", "messages": [{"role": "user", "content": f"Summarise in one paragraph:\n\n{text}"}], }, }) + "\n") ``` `custom_id` must be unique within the file, because results come back keyed by it. `body` is a normal chat-completion request; set `max_tokens` on a line if you need a limit other than the default. ## Upload and create ```python from privatemind import Client client = Client() # token and base URL from the workspace environment f = client.create_file(file=open("requests.jsonl", "rb"), purpose="batch") batch = client.create_batch( input_file_id=f.id, endpoint="/v1/chat/completions", completion_window="24h", ) print(batch.id, batch.status) # batch_... in_progress ``` `create_file` uploads the file as multipart form data and returns a `BatchFile` (`id`, `filename`, `bytes`, `purpose`, `status`, `created_at`). `create_batch` takes that file's `id` and returns a `Batch`. The optional `metadata=` is a dict of string keys and values returned on the batch object. The [Batch API page](https://docs.privatemind.com/batch.html) has its size limits. ## Wait for the batch ```python batch.wait() # blocks until terminal, refreshes the object in place ``` `Batch.wait()` polls while `status` is one of the four live values: `validating`, `in_progress`, `finalizing` or `cancelling`. It stops on the four terminal ones: `completed`, `failed`, `expired` or `cancelled`. Each poll updates the object in place, and `wait()` returns the batch itself. To classify a status yourself, import the two sets as `ACTIVE_BATCH_STATUSES` and `TERMINAL_BATCH_STATUSES`. - `poll_interval=` is the starting number of seconds between polls (default 5). It doubles each poll, up to 30 seconds, or up to your starting interval if you set one higher than 30. Zero or a negative value raises `ValueError`. - `timeout=` limits the wait, in seconds. `None`, the default, waits forever. When it runs out, `wait()` raises Python's built-in `TimeoutError` with the last status it saw in the message. - `on_unknown_status=` decides what happens if the server reports a status in neither set, which means your SDK is older than the platform. The default, `"raise"`, raises `UnknownStatusError` (carrying `.batch_id` and `.status`). It raises because the batch may still be running, and treating it as finished could let `results()` return a partial file. Pass `"return"` and `wait()` returns with a warning. A timeout stops only the waiting: the batch keeps running on the server. Wait again, or stop it with `cancel_batch`: ```python try: batch.wait(timeout=3600) except TimeoutError: print("still running:", batch.status, batch.request_counts) ``` ## Read the results ```python summaries = {} if batch.output_file_id: for row in batch.results(): body = row["response"]["body"] summaries[row["custom_id"]] = body["choices"][0]["message"]["content"] if batch.error_file_id: for row in batch.error_results(): print("failed:", row["custom_id"], row["error"]) ``` `results()` downloads the output file and yields one plain dict per JSONL line (`custom_id`, `response`, `error`), as written in the file. `error_results()` does the same for the error file, which holds the requests that failed. Output order is not guaranteed, so collect by `custom_id`. Both download the whole file before reading it, and a malformed line raises `PrivatemindError`. What each raises: - Before the batch is terminal, both raise `PrivatemindError` telling you to `wait()` first. - A terminal batch with no output file means no request succeeded. If the whole input file was rejected, `results()` and `error_results()` both raise and point you to `batch.errors`, which says why. Otherwise `results()` raises and points you to `error_results()`. - A terminal batch with no error file had no request fail, and `error_results()` yields nothing. On a cleanly completed batch, the `if batch.error_file_id:` check in the example above is optional. To keep the raw file itself, download its bytes by id: ```python raw = client.get_file_content(batch.output_file_id) with open("batch-output.jsonl", "wb") as fh: fh.write(raw) ``` ## The batch lifecycle A batch starts `in_progress` and ends in one of the four terminal statuses, named as in the [Batch API](https://docs.privatemind.com/batch.html): - `completed`: finished. `output_file_id` is set, and `error_file_id` too if any request errored. - `failed`: the input file was rejected as a whole; the diagnosis is on `batch.errors`. - `expired`: the 24 hour window ran out first. Finished requests are in the output file, the rest in the error file. - `cancelled`: you cancelled it. A batch with requests in flight finishes them first and shows `cancelling` meanwhile. Completed work is billed and stays readable in the output file. Progress is in `request_counts` (`total`, `completed`, `failed`). Cancelled and expired requests count toward `total` only, never `failed`. Check it with `get_batch`: ```python b = client.get_batch(batch.id) print(b.status, b.request_counts.completed, "of", b.request_counts.total) ``` ## Pagination `list_batches` returns one page as a plain list. It takes `limit=` (1 to 100) and `after=`, the id of the batch to start after. Pass `after=page[-1].id` for each next page until a page comes back empty: ```python batches = [] page = client.list_batches(limit=100) while page: batches.extend(page) page = client.list_batches(limit=100, after=page[-1].id) ``` The server can't filter by status, so filter in your code: ```python in_progress = [b for b in batches if b.status == "in_progress"] ``` Called with neither `limit=` nor `after=`, `list_batches()` raises `PrivatemindError` if the server reports the list was truncated. The error tells you to pass `after=` for the next page. With either argument, the call returns the page without raising. ## Handling errors Every SDK exception subclasses `PrivatemindError`; the [SDK overview](https://docs.privatemind.com/sdk.html) has the hierarchy. The read calls (`get_batch`, `list_batches`, `get_file_content` and the polling inside `wait()`) retry temporary server errors on their own. The write calls (`create_file`, `create_batch`, `cancel_batch`) are never re-sent for you, so wrap them in your own retry where it matters: ```python import time from privatemind import RateLimitError for attempt in range(5): try: batch = client.create_batch( input_file_id=f.id, endpoint="/v1/chat/completions", completion_window="24h", ) break except RateLimitError as exc: time.sleep(exc.retry_after or 2 ** attempt) ``` `RateLimitError.retry_after` holds the server's `Retry-After` header, or `None` if it sent none. ## Limits and retention The server enforces the request count, file size and rate limits, so they can change without an SDK upgrade. A rejected request raises a normal SDK exception. See [limits and retention](https://docs.privatemind.com/batch.html#limits-and-retention) for the current numbers. > [!NOTE] > Batch input and output files hold your content and are kept on the server for up to 29 days. Batch is not eligible for zero-data-retention (ZDR) arrangements. ## Where next - [Batch API](https://docs.privatemind.com/batch.html): the JSONL line format, the batch object, limits, and pricing. - [Files](https://docs.privatemind.com/files.html): the endpoints behind `create_file` and `get_file_content`. - [SDK overview](https://docs.privatemind.com/sdk.html): configuration, authentication, and the error hierarchy.