01
Price a credit
Decide what one credit is worth in dollars. $0.02, $0.10, $2, whatever fits your pricing.
Five ways an AI bill goes wrong. One rule for each.
Every fix below is the same three moves: filter the calls, set a limit, pick what happens at the line. No proxy. No prompt text leaves your servers.
Free tier
Cap what the whole free tier can cost you. However many accounts sign up.
Per-user caps are the intuitive control and the wrong unit. One person mints fake free accounts, each one gets its own fresh budget, and no single user’s limit ever trips. This is a Sybil attack. A $100 free-tier cap spread over 1,000 accounts is a $100,000 exposure. Pool the budget instead: one shared counter across every free user. An attacker can multiply identities. They cannot multiply the pool.
Per-user cap alone
A fresh budget per account
The limit every team reaches for first.
$5 a month, per free user
Feels safe. It is the wrong unit.
Every new account starts at zero
The cap resets itself for free, once per signup.
1,000 accounts, up to $5,000
Your exposure scales with how many accounts they can register.
No single budget ever trips
Each identity stays comfortably under its own limit.
Paid users unaffected
The one thing it does get right.
Shared free-tier pool
One counter for the whole tier
Apply the limit to everyone matching, not to each user.
$25 a day, across all free users
One budget, however many accounts exist.
Ten accounts or ten thousand
The pool is the same size either way.
Layer a per-user cap underneath
A $1 a day inner cap stops one account draining the pool alone.
Paid traffic is untouched
As long as your app sets
paid_planon every call.Starts in dry-run
Watch what it would have paused before it pauses anything.
It bounds the loss; it does not detect the attack. TokenPolice reads only what your app sends it. It never sees IP addresses or devices, and cannot tell a fake account from a real one. Pair the pool with your own signup defences. What the pool guarantees is the ceiling: a flood becomes a bounded daily cost instead of an open-ended bill.
A shared pool is shared. Once it trips, genuine free users pause too, until the window resets. Set it high enough to cover a normal day of real free usage.
Plans and credits
Ship AI credits without building a metering system.
You want to sell “50 credits on Free, 500 on Pro.” Normally that means a usage table, counters, scheduled resets and enforcement code in every call path. And it counts requests, not what those requests actually cost. In TokenPolice a credit is just a dollar amount. Price a credit at $0.02, and Pro’s 500 credits is one rule: $10.00 a month, per user.
Any period you sell on
Calendar windows reset on the UTC boundary: hourly, daily, weekly, monthly or lifetime. For credits that belong to one conversation, a sliding window resets after 5 to 1,440 minutes with no spend.
Re-price without a deploy
Change what a credit is worth, or a plan’s allowance, in the dashboard. Rules reach the firewall in seconds over the SDK’s live stream. Nothing to restart in your app.
Block, or land softly
At the line, stop the call and catch TokenPoliceBlockedError, or reroute to a cheaper model so the feature keeps working.
The ledger comes free
The metadata that enforces the limit also reports it: credits per call, per session, per hour, per day, per user, per plan.
1 credit = $0.02
- Free50 credits$1.00 / mo
- Pro500 credits$10.00 / mo
- Scale5,000 credits$100.00 / mo
alex@example.com · Pro
412 / 500 credits
$8.24 of $10.00
credit 501rerouted to gpt-4o-mini
02
Write one rule
IF paid_plan is pro → APPLY TO each user_id → $10.00 monthly → block or reroute.
03
Change it any time
A new allowance, a new credit price, a new tier. Edit it in the dashboard; your app never changes.
Runaway agents
A stuck run will not stop itself.
A budget rule limits spend over time. A loop is a different failure: one run, one trace, going in circles. It does not crash and it does not time out. It just keeps asking the same model the same question until somebody notices. A user report on Google’s developer forum describes a single call that ran for 37 hours. A developer posted a $700 overnight bill from a single retry loop.
Exact prompt repeat
Watches the recent calls in one run for a cycle at the head of the sequence: x, x, x, or x, y, x, y. Three identical calls in a row trip it by default. It is an exact match, so it is close to false-positive free. Window 20 calls, cycle length up to 3.
Per-trace hard cap
A ceiling on a single run in calls, dollars or tokens. 25 calls, $5.00 or 200,000 tokens by default. It makes no inference at all; it is a counter against a number, and it is the backstop under everything else.
Protected on day one
Every new project starts with both detectors already watching, one exact-repeat rule and two hard caps, in alert-only mode. Blocking is something you opt into.
One run cannot hurt another
Loop rules key off the trace, so a single user’s runaway loop never touches anyone else’s traffic.
It never reads your prompts
Detection runs on a fingerprint: role, type, length and a content hash for each message segment. Two calls match only when the prompt is identical byte for byte.
The trigger is approximate: the call that crosses is allowed, and a later call in the same run is stopped.
Cheaper-model routing
Downgrade the model instead of denying the user.
Blocking a paying customer is a support ticket. A reroute rule swaps the model before the call is dispatched. Your code still asks for gpt-4o, TokenPolice serves gpt-4o-mini, and the response comes back as normal. GPT-4o is roughly 16× the price of GPT-4o-mini per output token, so the traffic you downgrade is the traffic that stops mattering to your bill.
Always, or only when it is tight
Unconditional: every call on a given plan runs on the cheaper model. Budget-gated: the normal model until the pool crosses your limit, the cheaper one for the rest of the window.
Nothing changes in your request code
The SDK rewrites the model before dispatch. Read response.model if you want to tell the user which model served them.
It will not unpin your model
If the target resolves to the model you already asked for, say a rule naming claude-haiku-4-5 against your pinned dated snapshot, the rule is skipped rather than silently unpinning you.
Watch it before you switch it on
In dry-run the dashboard counts every call the rule would have rerouted, so you size the change before it reaches a user.
Swaps stay within the same provider. Reroute works by rewriting the model in the request body, so it needs a call that hands the SDK one — LangChain, LangGraph, LlamaIndex, Pydantic AI and the Vercel AI SDK do not. There the swap is refused, the call runs on the model you asked for, and the Routing tab lists it as a rejected reroute.
Cost per user, session, workflow
Know what a user or a session costs before the invoice does.
Most LLM tools trace prompts and score answer quality. TokenPolice measures money. Every call is priced per unit, every dollar decomposes to unit × quantity × rate, and the metadata that enforces your budgets is the same metadata you slice.
28 billable units, priced apart
Not just input and output. Prompt-cache reads, cache writes split by five-minute and one-hour TTL, reasoning tokens, audio tokens and audio seconds, image tokens and image pixels, TTS characters, OCR pages, rerank units, web searches, batch and priority tier multipliers, gateway fees.
Eight named cost drivers
When a workflow gets more expensive, the dashboard ranks why: more calls per run, longer prompts, longer responses, more reasoning tokens, cache reuse dropped, more tool calls per run, a shift to pricier models, or a provider price rise.
Four drift gauges
Prompt size, response size, turns per run and tools per run, each placed against that workflow’s own rolling 30-day baseline.
Line items, not a lump sum
Open any span in any agent run and read unit, quantity, rate and cost to six decimals, with the gateway fee on its own row.
Slice it by anything
User, plan, model, provider, workflow, session, trace, span, tool, modality, endpoint, service tier, and any custom label your app attaches. Grains from per-minute to per-week, always against the matching previous period.
We flag what we could not price
Calls the provider returned no usage for, or that have no price record, are marked and listed separately rather than quietly counted as $0. Your provider may still have billed you.
How every rule works
Five problems, three moves.
Filter
Which calls the rule watches. user_id, paid_plan, operation, or any label your app attaches.
Limit
How much they are allowed. Dollars, tokens or requests, over an hour, a day, a week, a month, a lifetime, or a sliding window.
Action
What happens at the line. Block the call, notify your team, or reroute to a cheaper model.
Every block and reroute rule starts in dry-run. It records what it would have done, and acts only once the rule and your app both say enforce. Notify has no dial: it fires whenever the rule matches.
Put a limit on your first user today.
Free up to 10,000 guarded requests a month. No credit card. The next runaway loop on someone else's timeline, not yours.
FAQ
Common questions
No. You add two statements at startup, an import and init(), and TokenPolice instruments the provider SDKs you already use. Your create() calls stay exactly as they are.
No. Only token counts, model names, cost metadata and the identifiers you attach are sent. Loop detection works on a hash and a length per message segment, never the text.
Your call goes through. Every check fails open by design, so budget enforcement is a safety net and never a dependency of your product.
Yes. Every block and reroute rule is created in dry-run and only records what it would have done. It acts once the rule and your app are both set to enforce.
Yes. Rules are evaluated in priority order, so a per-user credit allowance, a free-tier pool ceiling and a per-trace loop cap can all be live at once.