# Streaming Server-Sent Events for token-by-token responses, on chat completions, the Messages API and the Responses API. Set `"stream": true` on a chat completions request and the API sends the response as Server-Sent Events (SSE) while the model writes it. The [Messages API](https://docs.privatemind.com/messages-api.html) and the [Responses API](https://docs.privatemind.com/responses-api.html) stream too, each with its own events: see [Messages API](https://docs.privatemind.com/streaming.html#messages-api) and [Responses API](https://docs.privatemind.com/streaming.html#responses-api) below. ```bash curl -N "https://api.privatemind.com/v1/chat/completions" \ -H "Authorization: Bearer $PMIND_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "fast", "messages": [{"role": "user", "content": "Write a haiku about TCP."}], "stream": true }' ``` `curl -N` disables curl's output buffering so chunks print as they arrive. ## Wire format Each event is a `data: ` line: ```text data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"},"index":0}]} data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":"Packets"},"index":0}]} ... data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":28,"total_tokens":40}} data: [DONE] ``` - **`choices[].delta.content`** is the next piece of text. Concatenate the pieces to build the full response. - **The last chunk before `[DONE]` carries `usage`.** PrivateMind always requests `stream_options.include_usage`, so you get token counts at the end without a second call. - **`data: [DONE]`** ends the stream. It isn't JSON: treat it as an end marker. ## Keepalive pings The API sends an SSE comment every 10 seconds while the model has nothing new to send, such as during a slow first token or long reasoning: ```text : ping ``` Comments carry no data and never appear inside an event. Standard SSE parsers, such as `EventSource` and the OpenAI SDKs, discard them, so most clients need no change. If you parse the stream yourself, ignore any line that starts with `:`. Two things to expect: - **A stream can open with a ping.** If the model hasn't produced its first token yet, the first bytes on the wire may be `: ping`, before any `data:` line. - **Pings stop idle timeouts. They don't stop buffering.** Proxies and load balancers between you and the API won't close a connection that looks idle, but a buffering proxy still breaks streaming (see the warning below). ## SDK examples ```python from openai import OpenAI client = OpenAI(base_url="https://api.privatemind.com/v1", api_key="PMIND...:...") stream = client.chat.completions.create( model="", messages=[{"role": "user", "content": "Write a haiku about TCP."}], stream=True, ) for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) ``` ## Reasoning output Reasoning models stream their chain-of-thought in a **separate field**, `delta.reasoning`, and `delta.content` holds only the final answer, with no inline `…` wrapper. A non-streaming response carries the same text in `message.reasoning`. ```text data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"reasoning":"Let me work through"},"index":0}]} data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"reasoning":" the proof step by step…"},"index":0}]} data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":"The answer is 42."},"index":0}]} ``` Show or collapse `delta.reasoning` as you like, and concatenate `delta.content` for the answer. A model that returns reasoning lists `reasoning` in [`supported_parameters`](https://docs.privatemind.com/models.html#supported_parameters). If it also lists `reasoning_effort`, you can switch thinking on or off per request with [`reasoning_effort`](https://docs.privatemind.com/chat-completions.html#reasoning-effort). Reasoning is returned by default. To leave it out, send `"include_reasoning": false`: ```bash curl -s "https://api.privatemind.com/v1/chat/completions" \ -H "Authorization: Bearer $PMIND_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "reasoning", "messages": [{"role": "user", "content": "What is 17 × 23?"}], "include_reasoning": false }' ``` The model still thinks before it answers, and the reasoning tokens still count in `usage.completion_tokens`. Only the response changes: there is no `message.reasoning` text, and no `delta.reasoning` chunks on a stream. Use this switch on a thinking-only model, where `reasoning_effort: "off"` returns `400`. `include_reasoning` works on `/v1/chat/completions`, including chat lines in a [batch](https://docs.privatemind.com/batch.html), for models that list `include_reasoning` in `supported_parameters`. The Messages API ignores it and uses `thinking: {display: "omitted"}`: see [Messages API → Thinking](https://docs.privatemind.com/messages-api.html#thinking). Reasoning is unfiltered model output. It can echo your system prompt and it can state things that are not true, so decide deliberately whether your users see it. See [Shared responsibility](https://docs.privatemind.com/shared-responsibility.html#reasoning-output). > [!NOTE] > PrivateMind uses the field name `reasoning`, as OpenRouter and Ollama do. Some tools, such as Cline and OpenCode, look for DeepSeek's `reasoning_content`. For those it's usually cleaner to turn thinking off with `reasoning_effort: "off"` where the model allows it, since long reasoning traces add noise to tool-calling loops anyway. On a thinking-only model, send `include_reasoning: false` so the `reasoning` deltas never arrive. ## Messages API `POST /v1/messages` with `"stream": true` answers with the Anthropic event protocol, which the Anthropic SDKs and Claude Code parse natively. ```bash curl -N "https://api.privatemind.com/v1/messages" \ -H "x-api-key: $PMIND_KEY" \ -H "anthropic-version: 2023-06-01" \ -H "Content-Type: application/json" \ -d '{"model": "fast", "max_tokens": 1024, "stream": true, "messages": [{"role": "user", "content": "Write a haiku about TCP."}]}' ``` Every event is a named `event:` line followed by its `data:`: ```text event: message_start data: {"type":"message_start","message":{"id":"msg_abc","type":"message","role":"assistant","content":[],"model":"fast","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0}}} event: content_block_start data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"A haiku is 5-7-5..."}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"privatemind"}} event: content_block_stop data: {"type":"content_block_stop","index":0} event: content_block_start data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}} event: content_block_delta data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Packets"}} event: content_block_stop data: {"type":"content_block_stop","index":1} event: message_delta data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"input_tokens":12,"output_tokens":28,"cache_creation_input_tokens":0,"cache_read_input_tokens":0}} event: message_stop data: {"type":"message_stop"} ``` - **Blocks arrive in order**: thinking, then text, then any `tool_use` blocks, each opened by `content_block_start` and closed by `content_block_stop`. A tool call's arguments stream as `input_json_delta` events. - **`message_start` comes first**, before the model has produced anything, so a client sees the stream open at once. - **`message_delta` carries the stop reason and the usage** for the whole turn. - **Keepalives are real events.** Where chat completions sends a `: ping` comment, the Messages API sends `event: ping`, which the SDKs ignore. - **A stream that fails ends with `event: error`**, never with `message_delta`, so a cut-off answer is never mistaken for a complete one. This includes a model that goes back to a tool call after starting other output: that call's `tool_use` block has already stopped with incomplete input. Don't run it; retry the request. See [Errors](https://docs.privatemind.com/errors.html#messages-api). ```python import anthropic client = anthropic.Anthropic(base_url="https://api.privatemind.com", api_key="PMIND...:...") with client.messages.stream( model="", max_tokens=1024, messages=[{"role": "user", "content": "Write a haiku about TCP."}], ) as stream: for text in stream.text_stream: print(text, end="", flush=True) ``` ## Responses API `POST /v1/responses` with `"stream": true` answers with the Responses API events, which the OpenAI SDKs and Codex parse natively. ```bash curl -N "https://api.privatemind.com/v1/responses" \ -H "Authorization: Bearer $PMIND_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "fast", "input": "Write a haiku about TCP.", "stream": true}' ``` ```text event: response.created data: {"type":"response.created","sequence_number":0,"response":{"id":"resp_c62db3cb3a67498f9f435315c9ca8faa","object":"response","status":"in_progress",...}} event: response.output_item.added data: {"item":{"content":[],"id":"msg_ad65c35f4eaa47a9a2432786c11e528a","role":"assistant","status":"in_progress","type":"message"},"output_index":1,"sequence_number":163,"type":"response.output_item.added"} event: response.output_text.delta data: {"content_index":0,"delta":" handshake forms\npack","item_id":"msg_ad65c35f4eaa47a9a2432786c11e528a","logprobs":[],"output_index":1,"sequence_number":166,"type":"response.output_text.delta"} event: response.completed data: {"type":"response.completed","sequence_number":173,"response":{"id":"resp_c62db3cb3a67498f9f435315c9ca8faa","object":"response","status":"completed",...}} ``` - **Items arrive one at a time**, in the order the model writes them: usually reasoning, then the message, then any function calls. Each is opened by `response.output_item.added` and closed by `response.output_item.done`. Reasoning streams as `response.reasoning_text.delta`, text as `response.output_text.delta`, and a call's arguments as `response.function_call_arguments.delta`. - **`response.created` and `response.in_progress` come first**, before the model has produced anything. - **The terminal event carries the whole response**, `usage` included: `response.completed`, `response.incomplete`, or `response.failed`. - **Keepalives** are `: ping` comments, as on chat completions, except that after 60 seconds without an event the next one is sent as a `response.in_progress` event. - **A final answer checked against a `text.format`** with tools on offer arrives in one delta, after the check. If it has to be generated again, the second pass streams as it is written. See [With tools on offer](https://docs.privatemind.com/responses-api.html#with-tools-on-offer). - **A stream that fails ends with `response.failed`**, never with `response.completed`, so a cut-off answer is never mistaken for a complete one. See [Errors](https://docs.privatemind.com/errors.html#responses-api). ```python from openai import OpenAI client = OpenAI(base_url="https://api.privatemind.com/v1", api_key="PMIND...:...") with client.responses.stream( model="", input="Write a haiku about TCP.", ) as stream: for event in stream: if event.type == "response.output_text.delta": print(event.delta, end="", flush=True) ``` The full event sequence is in [Responses API → Streaming](https://docs.privatemind.com/responses-api.html#streaming). ## When to use streaming - **Interactive UIs.** Show output as it arrives. - **Long generations.** Show progress while the model writes. For batch jobs or short responses, a non-streaming request is simpler. > [!WARNING] > If a proxy or load balancer sits between you and the API, turn off response buffering on that path. Nginx buffers by default: set `proxy_buffering off` for the route, or send `X-Accel-Buffering: no` on the response. ## Where next - [Chat completions](https://docs.privatemind.com/chat-completions.html) for the non-streaming variant and the full parameter list. - [Tool use](https://docs.privatemind.com/tool-use.html) for how tool calls surface inside a stream. - [Errors](https://docs.privatemind.com/errors.html) for how errors appear mid-stream.