# Where observability stops and enforcement starts > Tracing tools tell you what a call cost after it ran. A firewall decides whether it runs. What that changes in practice, and why you still want both. - URL: https://tokenpolice.ai/blog/observability-isnt-enforcement - Published: 2026-09-10 - Author: TokenPolice Team - Tags: observability, comparison LLM observability is one of the parts of AI infrastructure that already works well. LangSmith traces chains. Helicone proxies and logs. Datadog correlates LLM calls with everything else in your system. Arize and Braintrust run evals. Good tools, all of them. The question we get from teams already running one: I saw the spike on the dashboard, the alert fired, so why did the provider still bill me? Because the reporting loop runs after the call. The trace arrives, the cost is tallied, the user is attributed, and some time later a threshold trips. The money was spent before any of that happened. That shape is right for debugging a regression or deciding whether to roll back a prompt change. It will not stop a runaway agent. (Some of these tools are proxies, and a few offer request-time limits at the gateway. That is real enforcement and worth turning on. It is a different mechanism from the dashboard-and-alert loop, which is what most teams mean when they say they have cost visibility.) ## Deciding before the call TokenPolice runs inside your app, in the SDK, before the request leaves for the provider. It checks the rules on that project and comes back with one of three answers: allow, stop, or swap the model for a cheaper one on the same provider. A stopped call raises `TokenPoliceBlockedError` in your code and costs nothing, because the provider was never asked. Audit log, Denied tab: five DENIED rows in ENFORCE mode, models gemini-2.5-flash and claude-sonnet-4-6 *Five calls stopped in enforce mode. Gemini and Anthropic never saw them.* Two things keep that from being frightening to turn on. There are two switches, the SDK's firewall mode and each rule's own mode, and a call is only stopped when both say enforce. So you can leave the app in dry-run and spend a week reading what a rule *would* have done. And the check fails open: if it has no answer in two seconds (the default), the call proceeds. We would rather miss a block than take your product down. Only token metadata leaves your servers. No prompt text, no completions. [Data privacy](/docs/concepts/data-privacy) lists the fields. ## When enforcement stops being optional Observability you always want. Enforcement starts mattering once any of these is true: - Free-tier users can reach your LLM feature. - Your agents recurse or call tools. - The monthly provider bill is large enough that finance asks about it. - You allocate cost across tenants and need a ceiling per tenant. ## We are not a replacement for your tracing stack We store no prompt text and ship no eval tooling, so there is nothing here to migrate off LangSmith or Braintrust for. We do draw a span waterfall for the calls we meter, with prompt and response composition and a cost per span, because a number is hard to argue with when you cannot see the run that produced it. [Cost observability](/solutions#observability) covers what that view holds, and [why TokenPolice](/docs/why) draws the same line in the docs. If you want to try the enforcement half: read [dry-run vs enforce](/docs/concepts/dry-run-vs-enforce), put one rule in dry-run on your noisiest project, and check the Dry-run tab next week.