NeuralTrust | Platform for Agent Security.

Two catalog policies control volume and spend:

Catalog name	Slug	Group	What it does
Rate Limiter	`rate_limiter`	Traffic control	Max requests per sliding window.
LLM Budget	`token_rate_limiter`	Quota	Token or dollar budget over a time window; reject or downgrade when exceeded.

Both respect policy scope (Gateway-wide or Targeted consumers) and support Enforce / Observe (and throttle where the plugin allows).

Configure in the console

Open Policies → Catalog.
Choose Rate Limiter or LLM Budget.
Set Scope (mode + gateway-wide / targeted consumers).
Fill the form (details below).
Save. Verify with Playground or Analytics → Policy / Cost.

Rate Limiter

rate_limiter counts requests in a sliding window at pre_request.

Settings

Setting	Backend	Notes
Limit	`limit`	Max requests per window. Required.
Window	`window`	Duration string: `30s`, `1m`, `1h`. Required.
Retry-After	`retry_after`	Seconds returned when limited (default `60`).
Group by header	`group_by_header`	Optional sub-partition (e.g. `X-User-Id`).

{ "slug": "rate_limiter", "settings": { "limit": 100, "window": "1m" } }

Limited responses carry X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After.

LLM Budget

token_rate_limiter, labeled LLM Budget in the catalog, caps LLM usage by provider tokens or USD over a time window. It checks the budget at pre_request and accrues usage at post_response. This is the catalog quota policy for spend over time. Use tokens for raw usage or dollars for cost. Dollar mode uses the same rates as telemetry and Analytics: registry contract pricing first (override, else discount × catalog), then the models.dev catalog. A policy custom_pricing map (API-only) still wins over the registry when set.

Configure in the console

Policies → Catalog → LLM Budget.
Under Budget, set unit, max, and time window.
Under When limit is exceeded, choose behaviour (and downgrade target if needed).
Optionally add Per model limits.
Optionally open Advanced Settings (group-by header, counting).
Set mode and scope, then save.

The console uses a dedicated form (not the generic schema renderer).

Settings (UI ↔ backend)

Budget

UI	Backend	Default	Meaning
Unit	`unit`	Tokens	Tokens or Dollars.
Max	`aggregate.max`	—	Ceiling for the window. Tokens must be whole numbers; dollars may be fractional. Required (> 0).
Time window	`aggregate.time_window`	e.g. `1h`	Value + unit (Seconds / Minutes / Hours / Days). Backend raises windows below `60s` to `60s`.

The form shows a live summary, e.g. “Allow 1000 tokens per hour.”

When limit is exceeded

UI	Backend	Meaning
Reject request	`behavior_on_exceeded: reject`	Block the call (default).
Downgrade model	`behavior_on_exceeded: downgrade_model`	Rewrite `model` to a cheaper target.
Downgrade to	`downgrade_to`	Required when downgrading. Same provider as the requested model.

Per model limits (optional)

Tighter budgets for specific models. Match by slug or wildcard (e.g. claude-opus-*). Most specific pattern wins.

UI	Backend
Model	`rules[].model`
Max	`rules[].max`
Time window	`rules[].time_window`

Advanced

UI	Backend	Default	Meaning
Group by header	`group_by_header`	empty	Separate counter per header value (e.g. `X-User-Id` for per-end-user budgets).
Counting	`counting`	Total (input+output)	Total, Input only, or Output only.

Fields present in the plugin but not edited in the console (left at defaults / API-only) include stream usage injection, count cache reads, and custom pricing.

Runtime

At pre_request, estimate whether the request would exceed the remaining budget for the scope (and matching per-model rule, if any).
If over limit under Enforce:
- Reject → stop upstream with a budget error.
- Downgrade → rewrite the request model to Downgrade to and continue.
At post_response, accrue actual tokens (or estimated dollars) against the counter.
Under Observe, over-limit traffic is not blocked; decisions still appear on policy events for Activity / Analytics.

Example

{
  "slug": "token_rate_limiter",
  "settings": {
    "unit": "dollars",
    "counting": "total",
    "aggregate": { "max": 50, "time_window": "1d" },
    "behavior_on_exceeded": "downgrade_model",
    "downgrade_to": "gpt-4o-mini",
    "rules": [
      { "model": "gpt-4o*", "max": 20, "time_window": "1d" }
    ],
    "group_by_header": "X-User-Id"
  }
}

Token pool example:

{
  "slug": "token_rate_limiter",
  "settings": {
    "unit": "tokens",
    "counting": "total",
    "aggregate": { "max": 1000000, "time_window": "1h" },
    "behavior_on_exceeded": "reject"
  }
}

Choosing a scope

Scope	Typical use
Gateway-wide	Org-level spend ceiling.
Targeted consumers	Per-tenant or per-app quotas.
Group by header	Fairness inside one consumer (per end-user) without a policy per user.

Policies overview
Request size — payload size limits
Analytics — Cost and Policy tabs
Registries — contract rates for dollar budgets
Smart routing — route by Simple / Medium / Hard labels; pair with LLM Budget for spend ceilings

Introduction

Getting started

Resources

Endpoints

Routing

Policies

MCP

Observability

API reference

Rate limiting & budgets

Configure in the console

Rate Limiter

Settings

LLM Budget

Configure in the console

Settings (UI ↔ backend)

Budget

When limit is exceeded

Per model limits (optional)

Advanced

Runtime

Example

Choosing a scope

​Configure in the console

​Rate Limiter

​Settings

​LLM Budget

​Configure in the console

​Settings (UI ↔ backend)

​Budget

​When limit is exceeded

​Per model limits (optional)

​Advanced

​Runtime

​Example

​Choosing a scope

​Related

Configure in the console

Rate Limiter

Settings

LLM Budget

Configure in the console

Settings (UI ↔ backend)

Budget

When limit is exceeded

Per model limits (optional)

Advanced

Runtime

Example

Choosing a scope

Related