Is model routing safe for quality?
Last updated 2026-08-19
Routing is safe exactly when substitutions are earned and bounded, and unsafe when they are guessed. Safe routing means: quality bars pre-registered before any savings are measured, evidence sealed so it cannot be quietly rerun until it passes, escalation that re-serves refusals and validation failures on your requested model, fragile traffic never routed at all, and a per-request receipt naming what ran. Finest is built as exactly that machine, and traffic it has not proven serves your requested model verbatim.
How naive routing actually fails
The reason naive routing fails is measured, not theoretical: quality cliffs are input-dependent and invisible on easy inputs. In the measurements behind The Request Compiler, every candidate model was perfect on clean text; on hard inputs one dropped to 18% recall on a rotated scan, one mistyped an entire dense lab panel, and one returned nothing at all, a silent give-up undetectable without checks. Every failure appeared only where nobody was looking.
The classic failures are all silent. A cheaper model answers fluently but wrong, and nothing downstream checks. A quality bar gets defined after the results are in, shaped by the savings it needs to justify. A refusal from the cheap arm is returned to the user instead of escalating. And when an incident finally surfaces, nobody can list which requests were affected, so everything rolls back at once, including the savings that were real.
The machinery that makes it safe
Each piece exists to remove a place where optimism could hide. Together they change the failure economics: the worst case of a bounded router is your exact request, served as written, at list price.
- Pre-registered bars
- The quality threshold and the corpus are fixed before measurement. A bar chosen after the fact is a rationalization, not a bar.
- Sealed evidence
- Records are immutable and versioned, with the corpus hash and bounds published. Evidence that can be rerun until it passes is not evidence.
- Escalation
- A cheaper arm that refuses, or fails a machine-checkable validator, re-serves on the requested model. The failure mode costs latency, not correctness.
- Verbatim lanes
- Traffic whose failure a validator cannot catch cheaply, such as tool calls and schema-bound streams, rides untouched on the requested model.
- A ladder, not a switch
- In The Request Compiler, a plan earns authority in stages: observation, shadow comparison against the requested model, a frozen plan confirmed on a sealed set, a sticky canary with a circuit breaker, then active service. A model version bump or prompt edit invalidates authority until re-confirmed.
- Receipts
- Every request records which configuration served and under which evidence, so incidents scope to requests, not to the whole program.
Put the burden of proof on the router
The correct default for any routing vendor is distrust, resolved by artifacts. Ask to see the evidence bar for a task class before it serves you, the escalation behavior in writing, and a receipt from a real request. Finest publishes its bars, states its escalation rule, serves everything unproven verbatim, and prices itself so that being wrong costs it the fee: 25% of proven savings, or nothing.
Questions people ask
- Will routing degrade my hardest traffic?
- Not on a bounded router. Hard task classes fail their bars, so they never demote, and anything unproven serves the requested model verbatim. On Finest the honest receipts of classes that failed the bar are published alongside the ones that passed.
- What traffic should never be routed?
- Traffic whose failures a validator cannot cheaply catch: tool and function calls, schema-bound streaming, and anything where a plausible wrong answer is expensive. Finest serves these verbatim by design.
- How do I audit a substitution after the fact?
- By its receipt: the configuration that served, the evidence bar it cleared, and the saving against your requested model, resolvable at finest.so/verification.
In 2 minutes, start cutting your API spend without sacrificing quality. Free if you don’t save money.
No model markup. You pay the host’s rate. 25% of what it proves it saved on a request. No saving, no fee.