You cannot audit what you did not log, and most teams running multiple model providers log almost nothing useful — a request ID, a duration, maybe a token count scraped from an SDK response. Good LLM routing telemetry is not complicated, but it has to be decided before the traffic happens, because a column you did not write is a question you can never answer retroactively.
Not by a log parser afterwards. The thing making the call knows facts that never reach a log line, and reconstructing them later produces numbers that are approximately right, which is the worst kind. Log these:
Identity and timing: call ID, parent task ID (so a multi-step agent's calls group into one workflow), UTC timestamp, and duration split into time-to-first-token and total. The parent ID is the one people forget, and without it you can never compute cost per completed task — only cost per call, which nobody actually cares about.
The decision: the lane chosen, the task domain, the lanes that were eligible, and why this one won. Recording the alternatives is what lets you ask later whether the router was right, rather than only what it did. Without it, every routing question becomes unanswerable archaeology.
Economics: input tokens, output tokens, cached tokens if the provider distinguishes them, unit price at call time, and computed cost. Store the price you paid, not a reference to a price table — tables change and your history becomes retroactively wrong. This single decision is what makes a historical cost claim defensible a year later.
Outcome: a success flag, an HTTP status, and — the column that earns its keep — an explicit error class. Auth, rate limit, timeout, provider 5xx, malformed output, refusal, tool error. Never let an error land as null. And a retry-of pointer when the call is a retry, so retry cost is attributable rather than invisible.
The column that changes everything: error class. Without it, a failure rate is a number you can only stare at. With it, the same number becomes a ranked list of things to fix — and usually shows that one configuration issue accounts for the bulk of it.
Three reasons, none of them ideological. First, auditability: a claim you cannot recompute is a claim you are trusting, and a file on your disk can be queried by anyone you hand it to — including a customer's diligence team. Second, no sampling: hosted observability platforms sample at volume and bill by event, so the first thing teams do at scale is turn down the fidelity of exactly the data they later need. Third, latency and privacy: writing a row locally costs microseconds and ships nothing off the machine, which matters when the prompt itself is the sensitive object.
Ours is a single SQLite file with a routing_log table and a lane_posterior table, written on every call. That is the whole apparatus. Its value is that one script reading it can reproduce every performance and cost claim we make — which is a different property from having a nice chart, and a much more useful one. The broader instrumentation picture is in the agent observability guide.
Traffic by tier. How much work went local, cheap, and premium. This is the claim most vendors make and least often evidence. Ours, over the 89.9 days to 9 August 2026 and 1,469 logged calls: 95 local, 1,322 cheap, 21 premium — premium at 1.4% of traffic.
Actual versus counterfactual spend. Sum your real cost, then re-price the identical call log at flagship rates. Ours: $0.14 actual against $65.28 all-flagship. The comparison is only honest because it re-prices the same rows — same prompts, same volume, same window — rather than comparing against a hypothetical workload.
Failure distribution by class and lane. The unflattering one. Ours shows an overall 19.9% success rate, dominated by 345 HTTP 403 responses from one provider and 756 failures with no class attached. That query turned a vague sense that "things fail sometimes" into two named defects with a scoped fix — a transport-layer classifier and an auth circuit breaker so configuration errors stop being recorded as quality evidence.
Cost per completed task. Group by parent task ID, include retries, divide by tasks that met the success definition. This is the only cost number that maps to business value, and it is impossible without the parent ID you decided to log at the start.
Rows are small — a few hundred bytes without payloads. Do not store full prompts and completions in the same table by default; store hashes plus a pointer, and keep the bodies in a separate store with a shorter retention window and tighter access. That keeps the analytical table fast, keeps sensitive text out of the thing you hand to a diligence team, and lets you retain economics for years while retaining content for weeks.
Index on timestamp, lane, and task ID. Roll up to daily aggregates once the raw table gets large, keeping raw rows for the recent window where debugging happens. The whole thing is a few hundred lines of code and it is the difference between having evidence and having opinions — which is exactly the standard we argue buyers should apply, including to us, in how to verify AI cost savings claims.
Should I log full prompts and responses? Sometimes, separately, with a shorter retention window. Keep the analytical table to metadata and economics; keep bodies in a store with tighter access control. Mixing them makes the fast table slow and the sensitive data hard to govern.
Is SQLite really enough at volume? For per-call routing telemetry on a single node, comfortably — these are small append-mostly rows. If you outgrow it, the schema ports directly to Postgres or a columnar store; the decision that matters is which columns you captured, not which engine holds them.
How do I compute a counterfactual cost honestly? Re-price your actual logged calls — same prompts, same token counts, same window — at the alternative model's published rate. Anything else is comparing your workload against an imagined one, which is how savings figures get inflated without anyone lying.
Add the error-class column today. It is one field and it converts your failure rate into a fix list. QADIR OS writes this telemetry to a local database you own and can query yourself — join the early-access list.