Provider rate limits are per account, not per end user. Your noisiest free-tier user sits under the same ceiling as your entire platform, so they can spend everybody else's budget before lunch. The fix is a per-user cap, checked before the call goes out.
Four decisions, then the rule.
Pick the unit
A TokenPolice limit can be an amount in USD, in tokens, or in requests. Requests are easy to reason about and wrong as a cost control: one call to gpt-4o and one to gpt-4o-mini are not the same money. Tokens have the same problem the moment you change models. Dollars survive a model swap, and dollars are the thing you compare against what the user pays you.
One example policy:
free→ $5 per user per monthpro→ $50 per user per monthenterprise→ one shared pool for the org, no per-user cap
Those first two numbers are the defaults in our "Cap spend for free users" and "Cap spend for paid users" templates. Your own numbers will differ. The shape is the part worth copying.
Check before the call, not after the invoice
Anything that reads billing data at month end is a report. A control has to run in the request path, before the provider is called, which puts a latency budget on it.
Ours is bounded by a timeout: 2 seconds by default. If it expires, or the check fails for any other reason, the call proceeds anyway. That is fail-open, and it is deliberate in both SDKs. We would rather miss an enforcement than drop your user's request.
Decide what happens at the cap
Three actions ship. Block stops the request in the SDK before the provider is called, and your app gets a TokenPoliceBlockedError to render as "you've hit today's limit". Notify lets the call through and alerts you. Reroute swaps to a cheaper model from the same provider before dispatch.
Reroute has one caveat worth knowing before you plan around it: on LangChain, LangGraph, LlamaIndex, Pydantic AI and the Vercel AI SDK the call still runs on the original model, and the Audit log records it as Reroute rejected. See actions.
There is no single rule that notifies at 80%, reroutes at 90% and blocks at 100%. One rule carries one limit and one action. You build that ladder as three rules on the same filter with three different limits, or you skip it and write one Block rule, which is what most free tiers actually need.
Tag the calls
A budget is only as good as its attribution. The SDK reads four things off the workflow wrapper: user_id, paid_plan, session_id and metadata.<key>. Those are the same fields rules filter and group on, and the same fields the Users page ranks by, so a call you forgot to tag becomes user anonymous on plan free and lands in the wrong pool. Worth getting right on day one: identity.
The rule
Firewall & Budgets → New rule from template → Cap spend for free users.
Plan name free, $5 per user, resets monthly on the 1st.
New Block rules start in Dry-run. Leave one there for a few days, read the Dry-run tab to see who it would have stopped, then come back and promote it to Enforce. Saving is live immediately, no deploy.
The saved rule, one row. Note is is exact: free does not match Free.
What a block looks like afterwards, in the Audit log's Denied tab.
Per-user caps do not stop signup abuse. A hundred throwaway accounts each stay politely under $5 and together cost you $500, which is why the free tier also wants one shared pool across the whole plan (more on that). The full walkthrough is the multi-tenant budgets recipe. The free plan covers 10,000 guarded requests a month.