You let a coding agent loose on a task. It loops. It re-reads the same files, re-summarizes the same context, retries the same failing call. Forty minutes later you check the provider dashboard and the run cost more than the feature was worth. You had four tools that could have told you, and none of them stopped it.
This is the gap most people do not notice until it costs them. The tools in this space look interchangeable from the outside, but they sit in three different places in the request lifecycle and solve three different problems. Here is the honest breakdown of where each one fits.
The three categories
Observability (Langfuse, Helicone, LangSmith). These record what your LLM calls did after they happened. Traces, token counts, latency, cost per call, evals. They are excellent for understanding behavior over time and debugging quality. They live beside the request path: the call completes, then the data flows to the dashboard. They can alert you that a budget was crossed. They cannot reach back and block the call that crossed it, because by the time the trace exists, the call is already paid for.
Gateways (LiteLLM, OpenRouter, Portkey). These sit in the request path. Routing, credentials, fallback, caching and budgets depend on the product and configuration. LiteLLM now documents agent/session-level limits as well as key, user and team budgets. A gateway can therefore overlap with run-level control; inspect its enforcement boundary rather than assuming budgets only work over billing periods.
Pre-flight cost control (Runcap). Runcap is a local-first control layer for AI coding agents. For requests you route through its local gateway, its job is different from the other two: estimate the cost of a run before it starts, and enforce a configured cap that returns an HTTP 429 before a routed call is sent once spend would cross it. That cap applies only to routed requests, not to direct provider calls, subscriptions, or unrelated tools. It is built around the moment before the money is spent on a routed call, where the other two act after or on a billing window.
Side by side
| Capability | Observability (Langfuse / Helicone) |
Gateway (LiteLLM / OpenRouter) |
Runcap |
|---|---|---|---|
| Estimate a routed run's cost before it starts | No | No | Yes (as a range) |
| Cap on routed requests, enforced before send | No (alert only) | Key/team and supported agent/session budgets | Yes (HTTP 429 at cap) |
| Proof Gate: replay an AI PR's verification before merge | No | No | Yes (GitHub Action) |
| Post-run traces and analytics | Yes (their strength) | Gateway logging; features vary | Run report + truth labels |
| Multi-provider routing and fallbacks | No | Yes (their strength) | Proxies Claude and OpenAI |
| Local-first, no server required | Cloud or self-host | Self-host option | Yes (local-first) |
The point of the table is not that Runcap wins every row. It does not. Langfuse will out-trace it; LiteLLM will out-route it. The point is the rows about the moment before a routed call is sent: estimate a routed run's cost before it starts, and cap routed requests before they leave your machine. Those are the rows that decide whether a runaway loop on a routed agent costs you a dollar or a hundred, plus the Proof Gate row that decides whether an AI-generated PR can be trusted to merge.
How the cap on routed requests works
Runcap runs a small local gateway. You point your agent's base URL at it, set a cap, and run your agent as normal. Every call you route through the gateway flows through it. Before a routed call is forwarded upstream, Runcap prices it against the live model rate and checks the running total. If forwarding the call would cross your cap, it never goes to the provider: the gateway returns HTTP 429 and the spend stays at zero for that call. Your agent sees a budget error, not a surprise bill. This applies only to requests routed through the gateway, not to direct provider calls, subscriptions, invoices, cards, or unrelated tools.
A note on token compression. Coding agents read a file, change one line, then re-read it, so two near-identical copies travel in the prompt. In one measured OpenAI edited-file re-read, prompt tokens dropped from 1,186 to 737 - 37.9% fewer prompt tokens - while the model still answered correctly about the changed line. This is a single measured result, not a universal or guaranteed saving. The full proof and a script you can run yourself are in the repo.
The honest claim. Runcap does not promise an exact cost oracle. Agent runs are stochastic; nobody can tell you the penny-precise cost in advance. What it gives you is a range before the run and a configured cap on routed requests during it. Every number it reports carries a truth label: observed, calculated, provider_usage, or unknown. It tells you which numbers are measured and which are estimated, instead of pretending they are all the same.
So which should you use?
This is not a cage match. These tools stack.
- You want to understand and improve quality over time across many runs in production: use Langfuse or Helicone. That is what they are for.
- You serve many users or rotate many providers and need routing, fallbacks, and per-key billing limits: use LiteLLM or OpenRouter.
- You are a developer running a coding agent (Claude Code, Codex, Cursor) on your own key and you want to estimate what a routed run will cost and enforce a cap on routed requests, plus a Proof Gate before an AI-generated PR can merge: use Runcap. It is free, MIT-licensed, and local-first.
The most common real setup is a gateway for routing, observability for after-the-fact analysis, and Runcap in front of the agent you actually let run unattended. They answer different questions: what did it do, where did it go, and how much before I let it go.
Try it
Runcap installs in one line and runs locally:
npm install -g runcap
Source and docs are on GitHub, and there is a plain-language walkthrough on the Runcap page. Everything is free, MIT-licensed, and local-first.
Cost control is one half of the story. The newer half is the Proof Gate: a pinned GitHub Action (runcap ci --mode adjudicate) that reads policy and verification rules from a pull request's base commit, checks the candidate diff against allowed scope and protected paths, replays permitted changes in a clean base checkout, and returns PASS, BLOCKED, or HUMAN_APPROVAL_REQUIRED before an AI-generated PR earns merge eligibility. It is a CI-attested replay under a documented hardened GitHub profile, not an unspoofable or fully independent check, not cryptographic proof, and not a replacement for human review. Current Proof Gate scope: GitHub Actions on Node/npm repositories. For the full walkthrough, read why a green CI check is not proof, or see the Runcap page.
Primary documentation / comparison update
Reviewed September 13, 2026. Consult LiteLLM budgets and limits and Langfuse token and cost tracking. These are external product documents, not independent benchmark results.