Batch

The privatemind SDK wraps the Batch API in six flat calls and a Batch object that waits and iterates results for you. The flow is the same as over HTTP: upload a JSONL file of chat-completion requests, create a batch from it, wait for a terminal status, read the output. Batch work is billed at 50% of the normal token rate; the Batch API page has pricing and the server-side rules.

Everything on this page runs as-is inside a PrivateMind workspace: Client() picks up its token and base URL from the environment. Configuration, authentication, and the exception hierarchy live on the SDK overview.

The six calls

One-to-one with the HTTP endpoints, available as methods on Client and as module-level imports (from privatemind import create_batch). The module-level functions share one lazily created, process-wide client; pass client= to any of them to use an explicit Client instead.

Three arguments are validated locally and raise ValidationError before any request is made:

• purpose must be "batch" (create_file)
• endpoint must be "/v1/chat/completions" and completion_window must be "24h" (create_batch)
• limit must be between 1 and 100 (list_batches)

Build the input file

The input is JSONL: one JSON object per line, one line per request, in the shape the Batch API defines (custom_id, method, url, body). Build it with a loop of json.dumps calls:

import json

docs = {
    "doc-1": "First quarterly report text ...",
    "doc-2": "Second quarterly report text ...",
    "doc-3": "Third quarterly report text ...",
}

with open("requests.jsonl", "w") as fh:
    for custom_id, text in docs.items():
        fh.write(json.dumps({
            "custom_id": custom_id,
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "glm-5-2-nvfp4",
                "messages": [{"role": "user", "content": f"Summarise in one paragraph:\n\n{text}"}],
            },
        }) + "\n")

custom_id must be unique within the file: results come back keyed by it, not by line order. body is a normal chat-completion request; set max_tokens per line if you need a non-default limit.

Upload and create

from privatemind import Client

client = Client()   # token and base URL from the workspace environment

f = client.create_file(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.create_batch(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(batch.id, batch.status)   # batch_... in_progress

create_file uploads the file as multipart form data and returns a BatchFile (id, filename, bytes, purpose, status, created_at). create_batch takes that file's id and returns a Batch. The optional metadata= is a dict of string keys and values echoed back on the batch object; the Batch API page has its size constraints.

Wait for the batch

batch.wait()   # blocks until terminal, refreshes the object in place

Batch.wait() polls the server until status is one of the four terminal values: completed, failed, expired, or cancelled (cancelling is not terminal, so wait() keeps polling through it). Each poll refreshes the object's fields in place, and wait() returns the batch itself.

• poll_interval= is the initial number of seconds between polls (default 5). It doubles on each poll, capped at 30 seconds, or at your initial interval if you set one higher than 30. Zero or negative values raise ValueError.
• timeout= bounds the wait in seconds; None (the default) waits forever. On expiry wait() raises Python's built-in TimeoutError, with the last status it saw in the message.

A timeout only stops the waiting: the batch keeps running server-side. Wait again, or stop it with cancel_batch:

try:
    batch.wait(timeout=3600)
except TimeoutError:
    print("still running:", batch.status, batch.request_counts)

Read the results

summaries = {}
if batch.output_file_id:
    for row in batch.results():
        body = row["response"]["body"]
        summaries[row["custom_id"]] = body["choices"][0]["message"]["content"]

if batch.error_file_id:
    for row in batch.error_results():
        print("failed:", row["custom_id"], row["error"])

results() downloads the output file and yields one plain dict per JSONL line (custom_id, response, error), exactly as written in the file. error_results() does the same for the error file, which holds the per-request failures. Output order is not guaranteed, so collect by custom_id. Both download the whole file before parsing it line by line, and a malformed line raises PrivatemindError rather than being skipped.

What each raises:

• Before the batch is terminal, both raise PrivatemindError telling you to wait() first.
• A terminal batch with no output file means no request succeeded. If the input file was rejected as a whole, results() and error_results() both raise pointing at batch.errors, where the diagnosis lives. Otherwise results() raises and points you at error_results().
• A terminal batch with no error file recorded no per-request failures: error_results() yields nothing, so on a cleanly completed batch the loop above is safe even without the guard.

To keep the raw file itself, download its bytes by id:

raw = client.get_file_content(batch.output_file_id)
with open("batch-output.jsonl", "wb") as fh:
    fh.write(raw)

The batch lifecycle

A batch is created in_progress and settles into one of four terminal statuses, with the same names as the Batch API:

• completed: finished. output_file_id is set, and error_file_id too if any request errored.
• failed: the input file was rejected as a whole; the diagnosis is on batch.errors.
• expired: the 24 hour window elapsed first. Finished requests are in the output file, the rest in the error file.
• cancelled: you cancelled it. A batch with requests in flight drains them first and shows cancelling on the way; completed work stays billed and readable in the output file.

Progress lives on request_counts (total, completed, failed). Cancelled and expired requests count toward total only, never failed. Poll it with get_batch:

b = client.get_batch(batch.id)
print(b.status, b.request_counts.completed, "of", b.request_counts.total)

Pagination

list_batches returns one page as a plain list and takes limit= (1 to 100) and after= (a batch id cursor). Advance with after=page[-1].id until a page comes back empty:

batches = []
page = client.list_batches(limit=100)
while page:
    batches.extend(page)
    page = client.list_batches(limit=100, after=page[-1].id)

There is no server-side status filter; filter client-side:

in_progress = [b for b in batches if b.status == "in_progress"]

A bare list_batches(), with neither limit= nor after=, raises PrivatemindError when the server reports the listing was truncated, telling you to pass after= for the next page. Passing either argument means you have opted into paging, and the call returns the page without raising.

Handling errors

Every SDK exception subclasses PrivatemindError; the hierarchy is on the SDK overview. The read calls (get_batch, list_batches, get_file_content, and the polling inside wait()) retry transient server errors automatically. The write calls (create_file, create_batch, cancel_batch) are never re-sent automatically, so wrap them in your own retry where it matters:

import time
from privatemind import RateLimitError

for attempt in range(5):
    try:
        batch = client.create_batch(
            input_file_id=f.id,
            endpoint="/v1/chat/completions",
            completion_window="24h",
        )
        break
    except RateLimitError as exc:
        time.sleep(exc.retry_after or 2 ** attempt)

RateLimitError.retry_after is filled from the server's Retry-After header when present, and is None otherwise.

Limits and retention

Request count, file size, and rate limits are enforced by the server, not the SDK, so they can change without an SDK upgrade; a rejected request surfaces as a normal SDK exception. See limits and retention for the current numbers.

Where next

• Batch API: the JSONL line format, the batch object, limits, and pricing.
• Files: the endpoints behind create_file and get_file_content.
• SDK overview: configuration, authentication, and the error hierarchy.