openapi: 3.1.0 info: title: PrivateMind Model Servers API summary: Launch, manage and register model servers on your GPUs. version: 1.0.0 description: >- Use a personal API key, or the short-lived token of a service identity granted model servers. Inference-only, management and application keys are refused, and a read-only key can only read. Launching also needs compute and GPU access on the account. Call a registered server through api.privatemind.com. servers: - url: https://privatemind.com/api/v1/model-servers security: - bearerAuth: [] - serviceToken: [] tags: - name: Discovery - name: Servers - name: Registration paths: /catalog: get: tags: [Discovery] operationId: getModelServerCatalog summary: List catalog models and vLLM versions responses: '200': description: The catalog. content: application/json: schema: $ref: '#/components/schemas/CatalogResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' /allocations: get: tags: [Discovery] operationId: getModelServerAllocations summary: List your org's GPU hosts responses: '200': description: The hosts your org owns. content: application/json: schema: $ref: '#/components/schemas/AllocationsResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' /volumes: get: tags: [Discovery] operationId: getModelServerVolumes summary: List your org's volumes responses: '200': description: Your org's volumes. metadata.name is the name a launch needs. content: application/json: schema: $ref: '#/components/schemas/VolumesResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' /images: get: tags: [Discovery] operationId: getModelServerImages summary: List the images your org pushed responses: '200': description: Your org's images, and how much of its storage they use. content: application/json: schema: $ref: '#/components/schemas/ImagesResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' /: get: tags: [Servers] operationId: listModelServers summary: List your org's model servers responses: '200': description: Your org's servers. content: application/json: schema: $ref: '#/components/schemas/ListResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' post: tags: [Servers] operationId: launchModelServer summary: Launch a model server requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LaunchRequest' responses: '201': description: The server was created and is starting. content: application/json: schema: $ref: '#/components/schemas/LaunchResponse' '400': $ref: '#/components/responses/Refused' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' '409': $ref: '#/components/responses/Refused' '422': $ref: '#/components/responses/Refused' /{name}: parameters: - $ref: '#/components/parameters/Name' get: tags: [Servers] operationId: getModelServer summary: Read a model server responses: '200': description: The server. content: application/json: schema: $ref: '#/components/schemas/GetResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' '404': $ref: '#/components/responses/Refused' delete: tags: [Servers] operationId: deleteModelServer summary: Stop a model server, release its GPUs and take it off the API responses: '200': description: The server was deleted. content: application/json: schema: $ref: '#/components/schemas/DeleteResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' '404': $ref: '#/components/responses/Refused' /{name}/gateway-register: parameters: - $ref: '#/components/parameters/Name' post: tags: [Registration] operationId: registerModelServer summary: Put a server on the API requestBody: required: false content: application/json: schema: $ref: '#/components/schemas/RegisterRequest' responses: '200': description: The server is registered. content: application/json: schema: $ref: '#/components/schemas/RegisterResponse' '400': $ref: '#/components/responses/Refused' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' '404': $ref: '#/components/responses/Refused' '409': $ref: '#/components/responses/Refused' /{name}/gateway-unregister: parameters: - $ref: '#/components/parameters/Name' post: tags: [Registration] operationId: unregisterModelServer summary: Take a server off the API and leave it running responses: '200': description: The server is not registered. content: application/json: schema: $ref: '#/components/schemas/RegisterResponse' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/Refused' '403': $ref: '#/components/responses/Refused' '404': $ref: '#/components/responses/Refused' components: securitySchemes: bearerAuth: type: http scheme: bearer description: A personal API key, access key id and secret joined by a colon. serviceToken: type: http scheme: bearer bearerFormat: JWT description: >- A service identity's short-lived token, from POST https://privatemind.com/api/auth/service/token with client_id and client_secret. The identity needs the model servers grant. parameters: Name: name: name in: path required: true description: The server's name, as returned when it was launched. schema: type: string responses: Unauthorized: description: A key that isn't valid, or no key on anything but a launch. content: application/json: schema: $ref: '#/components/schemas/Error' Refused: description: The request was refused. content: application/json: schema: $ref: '#/components/schemas/Error' schemas: Error: type: object required: [success] description: A reason is in error or message. error is a string or an object. properties: success: type: boolean error: oneOf: - type: string - type: object additionalProperties: true message: type: string detail: type: object additionalProperties: true LaunchRequest: type: object required: [targetCluster, gpuCount, vllmVersion] description: Name exactly one of modelSlug, weightsVolume or weights. properties: targetCluster: type: string gpuCount: type: integer minimum: 1 maximum: 8 description: How many GPUs, or MIG slices in Shared mode. vllmVersion: type: string modelSlug: type: string weightsVolume: type: string description: A volume's full name from /volumes. weights: type: string description: A : from your org's model registry, pinned to its digest at launch. Up to 100 GiB. image: type: string description: Your own image, such as customers/your-org/vllm:my-build. Needs gpuHost and gpuIndices. weightsSubPath: type: string volumes: type: array maxItems: 8 description: Your volumes to mount read-write, fixed at launch. items: type: object required: [volume, mountPath] properties: volume: type: string description: A volume's full name from /volumes. mountPath: type: string description: Absolute path in the server, for example /root/.cache/vllm. gpuHost: type: string gpuIndices: type: array items: type: integer gpuSharingMode: type: string enum: [Exclusive, Shared] gpuMemoryUtilization: type: string cpu: type: string description: Total vCPU, not per GPU. Defaults to 4. memory: type: string description: Host memory, for example 512Gi. sharedMemory: type: string description: Size of /dev/shm, for example 600Gi. Comes out of memory, so it can't be larger. args: type: object description: Extra vLLM flags without the leading dashes. additionalProperties: type: string displayName: type: string nativeApi: type: boolean description: No translation. True turns it on when the server is registered. Left out, it is off. LaunchResponse: type: object properties: success: type: boolean total: type: object properties: namespace: type: string name: type: string spec: type: object additionalProperties: true Condition: type: object properties: type: type: string status: type: string enum: ['True', 'False'] reason: type: string message: type: string ModelServer: type: object properties: metadata: type: object additionalProperties: true spec: type: object additionalProperties: true status: type: object properties: conditions: type: array items: $ref: '#/components/schemas/Condition' urls: type: object properties: openai: type: string ListedModelServer: allOf: - $ref: '#/components/schemas/ModelServer' - type: object properties: registered: type: boolean gatewayApiModelName: type: [string, 'null'] gatewayAliases: type: array items: type: string gatewayNativeApi: type: boolean GetResponse: type: object properties: success: type: boolean total: type: object properties: namespace: type: string name: type: string cr: $ref: '#/components/schemas/ModelServer' ListResponse: type: object properties: success: type: boolean body: type: array items: $ref: '#/components/schemas/ListedModelServer' total: type: object properties: namespace: type: string DeleteResponse: type: object properties: success: type: boolean total: type: object properties: namespace: type: string name: type: string deleted: type: boolean RegisterRequest: type: object properties: aliases: type: array maxItems: 3 items: type: string pattern: '^[a-zA-Z0-9]([a-zA-Z0-9._-]{0,61}[a-zA-Z0-9])?$' nativeApi: type: boolean description: No translation. Left out, a new registration takes the server's launch setting and a registered server keeps its setting. A value sent becomes the server's setting for later registrations. RegisterResponse: type: object properties: success: type: boolean total: type: object properties: name: type: string registered: type: boolean CatalogResponse: type: object properties: success: type: boolean body: type: array items: type: object additionalProperties: true properties: slug: type: string modelType: type: string compatibleVllmVersions: type: array items: type: string total: type: object properties: supportedVersions: type: array items: type: string AllocationsResponse: type: object properties: success: type: boolean total: type: object properties: org: type: string hosts: type: array items: type: object additionalProperties: true properties: host: type: string description: The host's name when it is dedicated to your org, otherwise an opaque ID you can pass back as gpuHost. cluster: type: string deployable: type: boolean server_type: type: [string, 'null'] total_gpus: type: integer dedicated: type: boolean description: True when your org holds the whole host. allocatable_cpu: type: [integer, 'null'] description: Present on a dedicated host. allocatable_memory_gib: type: [integer, 'null'] description: Present on a dedicated host. gpu_indices: type: array items: type: integer claims: type: object additionalProperties: true VolumesResponse: type: object properties: success: type: boolean body: type: array items: type: object additionalProperties: true properties: metadata: type: object additionalProperties: true spec: type: object additionalProperties: true ImagesResponse: type: object properties: success: type: boolean body: type: array items: type: object properties: image: type: string digest: type: string size_bytes: type: integer created_at: type: string format: date-time total: type: object properties: path: type: string used_bytes: type: integer quota_bytes: type: integer