Batch API · PrivateMind Docs

The Batch API runs large sets of chat-completion requests asynchronously. You upload a file of requests, create a batch from it, and collect the results within a completion window. Batch work is billed at 50% of the normal token rate in exchange for latency: results arrive within the window rather than immediately.

The shape matches OpenAI's Batch API, so the official openai client works against it unchanged.

The flow

Upload a JSONL file of requests with POST /v1/files (purpose: batch).
Create a batch from that file with POST /v1/batches.
Poll the batch with GET /v1/batches/{id} until it reaches a terminal status.
Download the output (and any error) file with GET /v1/files/{id}/content.

The input file

The input is a JSONL file: one JSON object per line, one line per request. Each line carries a custom_id you choose (unique within the file, used to match results back to requests), the method, the url, and the request body you would send to that endpoint.

JSON

{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "glm-5-2-nvfp4", "messages": [{"role": "user", "content": "Summarise: ..."}]}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "glm-5-2-nvfp4", "messages": [{"role": "user", "content": "Translate to French: ..."}]}}

url must be /v1/chat/completions. It is the only endpoint the Batch API runs.
custom_id must be unique within the file. Results are keyed by it, not by line order.
body is a normal chat-completion request. Set max_tokens per line if you need a non-default limit.

Upload the input file

curl -s "https://api.privatemind.com/v1/files" \
  -H "Authorization: Bearer $PMIND_KEY" \
  -F purpose=batch \
  -F file=@requests.jsonl

from openai import OpenAI

client = OpenAI(base_url="https://api.privatemind.com/v1", api_key=PMIND_KEY)
f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
print(f.id)  # file-...

The response is a file object; keep its id for the next step.

Create a batch

curl -s "https://api.privatemind.com/v1/batches" \
  -H "Authorization: Bearer $PMIND_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(batch.id, batch.status)  # batch_... in_progress

input_file_id (required): the id of an uploaded purpose: batch file.
endpoint (required): must be /v1/chat/completions.
completion_window (required): must be 24h.
metadata (optional): a map of string keys and values echoed back on the batch object. Up to 16 pairs; keys ≤64 chars, string values ≤512 chars.

The batch object

JSON

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "in_progress",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1781481050,
  "expires_at": 1781567450,
  "request_counts": { "total": 2, "completed": 0, "failed": 0 },
  "errors": null,
  "metadata": null
}

status is in_progress on create, then moves to a terminal status:

completed: the batch finished. output_file_id is set; error_file_id is set too if any request errored.
failed: the input file was rejected as a whole (for example, invalid JSONL). See errors on the object.
expired: the completion window elapsed before every request ran. Requests that did complete are in the output file; the rest are in the error file, coded batch_expired.
cancelling / cancelled: you cancelled it. In-flight requests drain first, then the batch settles to cancelled; if none are in-flight, it settles to cancelled directly without passing through cancelling.

request_counts tracks progress: total requests, completed succeeded, failed errored. Cancelled and expired requests count toward total only, never failed.

Check status

curl -s "https://api.privatemind.com/v1/batches/batch_abc123" \
  -H "Authorization: Bearer $PMIND_KEY"

import time

while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
print(batch.status)

Poll at a modest interval; there is no push notification. The batch is done when status is terminal.

Download results

The output file is JSONL, one line per succeeded request, keyed by your custom_id. Order is not guaranteed; match on custom_id.

curl -s "https://api.privatemind.com/v1/files/file-out-batch_abc123/content" \
  -H "Authorization: Bearer $PMIND_KEY"

import json

if batch.output_file_id:
    content = client.files.content(batch.output_file_id).text
    for line in content.splitlines():
        row = json.loads(line)
        print(row["custom_id"], row["response"]["status_code"])

Each output line carries a batch-scoped id, your custom_id, the HTTP response (its status_code, a request_id, and the chat-completion body), and error (null on success):

JSON

{"id": "batch_req_...", "custom_id": "req-1", "response": {"status_code": 200, "request_id": "req_...", "body": { "id": "chatcmpl-...", "choices": [ ... ] }}, "error": null}

Requests that errored are in the error file (error_file_id), same JSONL shape, with response null and error describing the failure. A batch whose requests all failed has no output file; read the error file instead.

Cancel a batch

cURL

curl -s -X POST "https://api.privatemind.com/v1/batches/batch_abc123/cancel" \
  -H "Authorization: Bearer $PMIND_KEY"

Cancellation drains in-flight requests rather than killing them, so the batch may pass through cancelling if any are in-flight; if none remain, it settles to cancelled directly. Work already completed is billed and available in the output file.

List batches

cURL

curl -s "https://api.privatemind.com/v1/batches?limit=20" \
  -H "Authorization: Bearer $PMIND_KEY"

GET /v1/batches returns one page. Advance with after=<last batch id> until a page comes back empty. There is no server-side status filter; filter client-side on status.

Limits and retention

Requests per batch: up to 10,000.
Input file size: up to 64 MB (the Files API ceiling).
Completion window: 24 hours. Requests not run by then expire.
Pricing: 50% of the normal per-token rate for the model, applied to work that actually ran.
Retention: the batch's input, output, and error files are retained for 29 days after the batch reaches a terminal status, then deleted. Download results you want to keep within that window.

Using the PrivateMind Python SDK

The PrivateMind SDK exposes the same flow with flat helpers and a Batch object that can wait and iterate results for you:

Python

from privatemind import Client

client = Client()  # base_url + token from the environment

f = client.create_file(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.create_batch(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
batch.wait()                     # poll until terminal
for row in batch.results():      # one dict per output line
    print(row["custom_id"], row["response"]["status_code"])

Where next

Chat completions for the request body each batch line carries.
Files for uploading input files and downloading results by id.
Budgets and limits for how the 50% batch rate lands against your budget.
Errors for status codes and error shapes.