A new model goes through the checks on this page before we offer it. The model register on the Models page shows where each model stands.
Licence
We review the model's licence and terms of use before we take it on. We keep a copy of both as they stood at that point.
Weights
We record the exact version of the weights: a single commit in the publisher's repository. We then read every file back from storage on a second machine and check it against the hash recorded for it in the publisher's repository at that commit. The model register shows the date of that check and how many files matched.
Other parts that run with the model are checked the same way, such as a draft model used to speed up generation (speculative decoding). We record the serving software by its image digest, which identifies one exact build.
Safety and capability tests
We test chat models on the exact build we serve. The safety tests cover:
- harmful requests, which the model should refuse
- over-refusal, where the model turns down a request it should answer
- adversarial prompts written to get past its safeguards
- harmful tasks given to the model as an agent with tools
- prompt injection, where instructions are hidden in tool results, text or images
- requests for help with cyber attacks
- honesty, when the model is pushed to say something it believes is false
- agentic misalignment, where a model acting as an agent works against the people it acts for, for example through blackmail or leaking data
The capability tests cover reasoning, maths, instruction following, tool calling, vision and long context. Vision tests apply only to models that accept images.
For models that are not chat models, such as embedding, speech, OCR and image models, the tests are a functional and performance check. Their safety tests are still being set.
If a test needs a judge, a judge model that we host ourselves grades the answers. A second model checks every answer the judge flags. Most other tests are scored by exact checks, such as matching the correct answer. Results are recorded as rates or scores, with 95% confidence intervals where the scoring gives one.
Serving checks
Before the tests run, we check the model on its serving setup. We try very long prompts, many requests at once, and each feature the model offers, such as tool calling, image input and structured output. Inputs it does not support must be refused.
Approval and records
Three senior roles approve a model before release: one for security, one for licence and compliance, and one for deployment. Each approval is recorded with the approver's name and the date.
We keep the evidence for each model, including the weights check and the test results, for seven years in storage with object lock. The accounts that write and read the evidence cannot change or delete it. Any removal for a legal or erasure demand is audited.
When we test again
We test a model again after any change to its weights, quantisation, serving software, speculative decoding setup or default reasoning effort.
Where each model stands
The model register shows, for each model, the date of its weights check, which tests have run and the date it was approved.
Where next
- Models for each model's entry in the register.
- Shared responsibility for the controls that stay with your application.