# Vision Multimodal chat: text plus images, in the OpenAI content-array shape or as Messages API blocks. A vision-capable model reads images sent with your text. Send the message `content` as an array of typed parts, OpenAI's standard multimodal format. The [Messages API](https://docs.privatemind.com/messages-api.html) takes images and documents as content blocks: see [Messages API](https://docs.privatemind.com/vision.html#messages-api) below. ```bash curl -s "https://api.privatemind.com/v1/chat/completions" \ -H "Authorization: Bearer $PMIND_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "vision", "messages": [{ "role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo..."}} ] }] }' ``` The `content` array can mix any number of `text` and `image_url` parts. ## Image sources - **Data URL**: `data:image/;base64,`. Use this when you have the image bytes locally. - **HTTPS URL**: a public URL, which the model must be able to fetch. Data URLs are more reliable. PNG and JPEG always work. WebP and GIF depend on the engine serving the model. ## Encoding a local file ```python import base64 from openai import OpenAI client = OpenAI(base_url="https://api.privatemind.com/v1", api_key="PMIND...:...") with open("photo.png", "rb") as f: b64 = base64.b64encode(f.read()).decode() resp = client.chat.completions.create( model="vision", messages=[{ "role": "user", "content": [ {"type": "text", "text": "Describe this."}, {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}, ], }], ) print(resp.choices[0].message.content) ``` ## Which models are vision-capable? A model that accepts images has `image_input: true` in its `capabilities` on `GET /v1/models`. Check that flag, not the model id. > [!NOTE] > Images use tokens: more for larger images, depending on how the model splits them into tiles. Resize before sending if you don't need full resolution. ## Messages API On `POST /v1/messages`, images and documents are content blocks in a user turn. Each block has a `source`: | Block | `source.type` | Notes | | --- | --- | --- | | `image` | `base64` | `media_type` plus the base64 `data` | | `image` | `url` | A public URL the model can fetch | | `image` | `file` | A `file_id` from the [Files API](https://docs.privatemind.com/files.html#messages-api); PNG, JPEG, GIF or WebP | | `document` | `text` | Plain text in `data`, with an optional `title` | | `document` | `content` | An array of text blocks | | `document` | `file` | A `file_id` holding text | ```python import base64 import anthropic client = anthropic.Anthropic(base_url="https://api.privatemind.com", api_key="PMIND...:...") with open("photo.png", "rb") as f: b64 = base64.b64encode(f.read()).decode() message = client.messages.create( model="vision", max_tokens=1024, messages=[{"role": "user", "content": [ {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": b64}}, {"type": "text", "text": "Describe this."}, ]}], ) ``` PDF documents are not supported and return `400`. Extract the text and send it as a `text` document. An image inside a `tool_result` works too: the model sees it as part of the next user turn. ## Where next - [Chat completions](https://docs.privatemind.com/chat-completions.html) for the underlying request shape. - [Messages API](https://docs.privatemind.com/messages-api.html) for the Anthropic-shape request. - [Models](https://docs.privatemind.com/models.html) to identify vision-capable models in your catalog. - [Tool use](https://docs.privatemind.com/tool-use.html) for combining vision with function calling.