# Fair share Share a busy model fairly between the people in your organization. When several people use one of your organization's dedicated models at once, the model can serve only so many requests at a time and the rest wait. Fair share decides who goes first. It does not refuse requests or cut answers short: it changes only the order of the requests that are waiting. When the model has room to spare, it makes no difference. > [!NOTE] > Fair share is available on models deployed for your organization alone; a model's settings say if it cannot use it. Org admins set it up on the **Inference** tab of the organization settings. ## How it decides Fair share adds up each person's recent use of the model as the work it took to serve them: by default, the prompt tokens the model had to process plus the tokens it generated. Prompt tokens served from the model's cache are free. Older use counts for less: after one usage window it counts half, after two a quarter, and so on. While the model is busy, the request from the person with the least recent use, relative to their weight, goes first. Someone with a weight of 200% is entitled to twice the share of someone at 100%. Over any busy stretch, everyone who keeps requests waiting gets a share in line with their weight. Someone coming back after a quiet spell has little recent use, so without a limit they could stay ahead of everyone for a long time. The burst cap limits that: a returning person leads the busy users for about the cap, then takes turns like everyone else. The method follows the virtual token counter in [Fairness in Serving Large Language Models](https://arxiv.org/abs/2401.00588) (Sheng et al., OSDI 2024), with older use fading over time. ## Shadow and enforce - **Shadow (measure only)** works out the order without applying it. Requests are served as before, and you can see what fair share would have done. Use it to check your settings against real traffic. - **Enforce** applies the order. Switch to **Enforce** at a quiet moment. Requests already waiting when you switch keep the place they had, so on a busy model they can wait behind newer requests for several minutes. ## Settings Leave a setting empty to use the default, which the form shows in the field. | Setting | What it does | |---|---| | Mode | **Weighted fair share** orders requests as described above. **Off** serves requests in the order they arrive. | | Enforcement | **Shadow (measure only)** or **Enforce**, as above. **Inherit** takes the platform default, shown under the field. | | Usage window (minutes) | How quickly past use fades: after this long it counts half. A longer window remembers heavy use for longer. | | Burst cap (minutes) | How long someone returning after a quiet spell can lead the busy users. 0 gives a returning person no head start at all. Weighted fair share only. | | Input (uncached), Cached input, Output | How each kind of token counts towards a person's use. By default uncached prompt tokens and output tokens count the same and cached prompt tokens count nothing. | ## User weights Weights are set per person, as a percentage: 100% is normal, 200% gets twice the share and 50% half. A weight applies to every model on weighted fair share in your organization. A change takes effect from that moment on. Past use isn't recounted, so a new weight never suddenly holds anyone back or lets them jump ahead. An application key counts as one person. If many people work through one app, its key shares a single person's portion unless you give it a higher weight. ## Watching it If your organization has a PrivateMind monitoring portal, its **Fair share** dashboard shows the mode in use, each person's share of the work, how long their requests took, and how busy the model was. ## What it does not do - It does not limit how much anyone can use. For that, see [Budgets & limits](https://docs.privatemind.com/budgets-and-limits.html) and [Rate limits & budgets](https://docs.privatemind.com/rate-limits.html). - It does not make a busy model faster. It changes the order of the requests that are waiting, not how many the model can serve. - It does not apply to models shared with other organizations, where the order is managed for you.