> ## Documentation Index
> Fetch the complete documentation index at: https://neuraltrust-92b43583-develop.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate limiting & budgets

> Limit request volume with Rate Limiter, and cap LLM spend over time with LLM Budget (token_rate_limiter) — tokens or dollars, aggregate or per-model.

Two catalog policies control volume and spend:

| Catalog name                      | Slug                 | Group           | What it does                                                                  |
| --------------------------------- | -------------------- | --------------- | ----------------------------------------------------------------------------- |
| **[Rate Limiter](#rate-limiter)** | `rate_limiter`       | Traffic control | Max requests per sliding window.                                              |
| **[LLM Budget](#llm-budget)**     | `token_rate_limiter` | Quota           | Token or dollar budget over a time window; reject or downgrade when exceeded. |

Both respect [policy](/trustgate/policies/overview) scope (**Gateway-wide** or **Targeted** consumers) and support **Enforce** / **Observe** (and throttle where the plugin allows).

***

## Configure in the console

1. Open **Policies** → **Catalog**.
2. Choose **Rate Limiter** or **LLM Budget**.
3. Set **Scope** (mode + gateway-wide / targeted consumers).
4. Fill the form (details below).
5. Save. Verify with **Playground** or
   [Analytics → Policy](/trustgate/observability/metrics) / **Cost**.

***

## Rate Limiter

**`rate_limiter`** counts requests in a sliding window at `pre_request`.

### Settings

| Setting             | Backend           | Notes                                             |
| ------------------- | ----------------- | ------------------------------------------------- |
| **Limit**           | `limit`           | Max requests per window. **Required.**            |
| **Window**          | `window`          | Duration string: `30s`, `1m`, `1h`. **Required.** |
| **Retry-After**     | `retry_after`     | Seconds returned when limited (default `60`).     |
| **Group by header** | `group_by_header` | Optional sub-partition (e.g. `X-User-Id`).        |

```json theme={null}
{ "slug": "rate_limiter", "settings": { "limit": 100, "window": "1m" } }
```

Limited responses carry `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`,
and `Retry-After`.

***

## LLM Budget

**`token_rate_limiter`**, labeled **LLM Budget** in the catalog, caps LLM usage by
**provider tokens or USD** over a time window. It checks the budget at `pre_request` and
accrues usage at `post_response`.

This is the catalog quota policy for spend over time. Use **tokens** for raw usage or
**dollars** for cost. Dollar mode uses the same rates as telemetry and Analytics:
[registry contract pricing](/trustgate/concepts/registries#contract-pricing) first
(override, else discount × catalog), then the models.dev catalog. A policy
`custom_pricing` map (API-only) still wins over the registry when set.

### Configure in the console

1. **Policies** → **Catalog** → **LLM Budget**.
2. Under **Budget**, set unit, max, and time window.
3. Under **When limit is exceeded**, choose behaviour (and downgrade target if needed).
4. Optionally add **Per model limits**.
5. Optionally open **Advanced Settings** (group-by header, counting).
6. Set mode and scope, then save.

The console uses a dedicated form (not the generic schema renderer).

### Settings (UI ↔ backend)

#### Budget

| UI              | Backend                 | Default   | Meaning                                                                                                       |
| --------------- | ----------------------- | --------- | ------------------------------------------------------------------------------------------------------------- |
| **Unit**        | `unit`                  | Tokens    | **Tokens** or **Dollars**.                                                                                    |
| **Max**         | `aggregate.max`         | —         | Ceiling for the window. Tokens must be whole numbers; dollars may be fractional. **Required** (> 0).          |
| **Time window** | `aggregate.time_window` | e.g. `1h` | Value + unit (**Seconds** / **Minutes** / **Hours** / **Days**). Backend raises windows below `60s` to `60s`. |

The form shows a live summary, e.g. “Allow **1000** tokens **per hour**.”

#### When limit is exceeded

| UI                  | Backend                                 | Meaning                                                          |
| ------------------- | --------------------------------------- | ---------------------------------------------------------------- |
| **Reject request**  | `behavior_on_exceeded: reject`          | Block the call (default).                                        |
| **Downgrade model** | `behavior_on_exceeded: downgrade_model` | Rewrite `model` to a cheaper target.                             |
| **Downgrade to**    | `downgrade_to`                          | Required when downgrading. Same provider as the requested model. |

#### Per model limits (optional)

Tighter budgets for specific models. Match by slug or wildcard (e.g. `claude-opus-*`).
Most specific pattern wins.

| UI              | Backend               |
| --------------- | --------------------- |
| **Model**       | `rules[].model`       |
| **Max**         | `rules[].max`         |
| **Time window** | `rules[].time_window` |

#### Advanced

| UI                  | Backend           | Default              | Meaning                                                                        |
| ------------------- | ----------------- | -------------------- | ------------------------------------------------------------------------------ |
| **Group by header** | `group_by_header` | empty                | Separate counter per header value (e.g. `X-User-Id` for per-end-user budgets). |
| **Counting**        | `counting`        | Total (input+output) | **Total**, **Input only**, or **Output only**.                                 |

Fields present in the plugin but not edited in the console (left at defaults / API-only)
include stream usage injection, count cache reads, and custom pricing.

### Runtime

1. At **pre\_request**, estimate whether the request would exceed the remaining budget for the
   scope (and matching per-model rule, if any).
2. If over limit under **Enforce**:
   * **Reject** → stop upstream with a budget error.
   * **Downgrade** → rewrite the request model to **Downgrade to** and continue.
3. At **post\_response**, accrue actual tokens (or estimated dollars) against the counter.
4. Under **Observe**, over-limit traffic is not blocked; decisions still appear on policy
   events for Activity / Analytics.

### Example

```json theme={null}
{
  "slug": "token_rate_limiter",
  "settings": {
    "unit": "dollars",
    "counting": "total",
    "aggregate": { "max": 50, "time_window": "1d" },
    "behavior_on_exceeded": "downgrade_model",
    "downgrade_to": "gpt-4o-mini",
    "rules": [
      { "model": "gpt-4o*", "max": 20, "time_window": "1d" }
    ],
    "group_by_header": "X-User-Id"
  }
}
```

Token pool example:

```json theme={null}
{
  "slug": "token_rate_limiter",
  "settings": {
    "unit": "tokens",
    "counting": "total",
    "aggregate": { "max": 1000000, "time_window": "1h" },
    "behavior_on_exceeded": "reject"
  }
}
```

### Choosing a scope

| Scope                  | Typical use                                                            |
| ---------------------- | ---------------------------------------------------------------------- |
| **Gateway-wide**       | Org-level spend ceiling.                                               |
| **Targeted consumers** | Per-tenant or per-app quotas.                                          |
| **Group by header**    | Fairness inside one consumer (per end-user) without a policy per user. |

***

## Related

* [Policies overview](/trustgate/policies/overview)
* [Request size](/trustgate/policies/request-size) — payload size limits
* [Analytics](/trustgate/observability/metrics) — Cost and Policy tabs
* [Registries](/trustgate/concepts/registries#contract-pricing) — contract rates for dollar budgets
* [Smart routing](/trustgate/routing/smart-routing) — route by Simple / Medium / Hard labels; pair with LLM Budget for spend ceilings