How verification works

We serve the model you asked for. We substitute a cheaper configuration only where a recorded test already proved that configuration does this exact job, at a quality bar fixed before the test ran. When a substitution holds and costs less, we charge 25% of the difference, on that request, as it is served. When we prove no saving, you pay us nothing.

Those four sentences are easy to say. The rest of this page is the machinery that stops them from being marketing: what the saving is measured against, what can never carry a fee, and which of the guarantees are database constraints rather than promises. If you would rather read the contractual version, it is section 9 of the terms, and the two do not disagree; the worked example, computed by the billing code itself, is on the pricing page.

One consequence is worth stating up front. If your current setup already wins, every receipt says $0 and you owe $0: the worst outcome available here is a precise, itemised audit of your model bill.

It is the request itself. The baseline for every request is the model that request names: what you asked for, priced from current published provider prices over the same metered tokens the request actually used. Your request sets the number we measure against, and nothing we recommend or serve can raise it. A recommendation can never become the baseline.

The arrival record is a disclosure, not a meter. The console still keeps a record of the models you arrived running: declared when your first gateway key is issued, or earned later when your own traffic serves a model through an observed window. That record is yours to confirm or leave alone, and neither choice moves a bill. It is never a fee input, and confirming it cannot raise what any request is billed against.

Charged at serve time, on proof. A fee exists only where we actually substituted a model and the substitution cost less than the model you asked for on that request. The receipt for every fee-bearing request shows both models and both prices; the saving is their difference over the same metered tokens, and the 25% share is debited as the request is served. A request served on the model you named is never fee-bearing.

Saying yes to us cannot raise it. The billing arithmetic takes the lower of what you were spending and what the configuration we recommended would have cost. Accepting a recommendation therefore cannot raise the number we bill against: if you spent $100, we recommended a $150 configuration, and your traffic actually cost $90, the eligible saving is $10, not $60. A pricier model joins your baseline only if you independently run it in production for a defined window.

Price cuts are not our savings. Comparisons are priced from current published provider prices. When a provider cuts its price, the cost of the model you asked for falls with it, and the difference is never billed as a saving, because you would have had that saving without us.

The whole law, applied per request at the moment it is served:

eligible_savings = max(0,
    min(origin_baseline_cost, approved_counterfactual_cost)
    − actual_fully_loaded_cost)

fee = 25% × eligible_savings   # per request, as it is served. No netting, no allowance.

Each request stands alone: there is no monthly aggregation in which a bad request can hide, and rounding is floored in your favour. The arithmetic lives in one module; receipts, statements and the pricing page’s worked example all call it and none re-derive it.

Never fee-bearing, by construction:

Two guards sit under all of this, in the database rather than in code that could regress. What you are debited can never exceed the cost of the request you asked for, so your worst case is exactly what you asked for at pass-through price. And a fee above 25% of a request’s saving is structurally unwritable. A dishonest fee row is not discouraged; it is refused by the table.

The unit is the full configuration. Not a model name: the provider, endpoint, pinned model version, host, quantization, service tier, prompt and schema revisions, reasoning effort, decoding settings and cache policy, together. Two hosts serving the same weights are two candidates, because they do not behave identically.

The bar is set before the test. Candidates are screened on a development split, and exactly one selected configuration advances to a confirmatory run on a frozen holdout it has never touched. If the holdout is too small to support the pre-registered bar, the run blocks rather than proceeding on a weaker one.

One look, then the data is spent. A given plan and corpus may be judged exactly once, keyed by content hashes rather than filenames, and a consumed holdout split is consumed forever. A failed confirmation publishes nothing. An abstained one publishes nothing either: abstain is not a pass with paperwork.

Judged blind, reproducibly. Most test items settle on deterministic graders. The residue goes to a judge panel that cannot see which configuration produced which answer, with seating derandomized from a hash so a rerun reproduces the same assignment bit for bit. Every consequential check is a named detector with a measured false-accept rate; the schema refuses an unnamed one.

In production, we hold the byte. We never emit a byte we might need to retract. Validators run before the response reaches you; when one refuses the cheap attempt, the model you asked for answers, the receipt records the escalation, and the detour is our cost. A request you mark with x-finest-no-demote is never substituted at all.

Passing once is not passing forever. Daily sentinels re-measure every live configuration against its frozen expectations, on a pre-registered statistical schedule with tripwires for sudden collapse. Drift retires an entry, and republication requires fresh, unconsumed data through the same ceremony, not a second read of the old evidence. Every entry also carries a hard expiry; past it, serving falls back to a pinned baseline of your original configuration.

Every request we serve returns a receipt naming the model you asked for and the model that served it, both prices, token counts, the saving, the fee, and what was actually debited, each money figure carrying its state. Receipts are append-only by database trigger, and the runtime role’s permission to update or delete them is revoked outright, so the record of what happened cannot be edited into a record of what should have happened.

You can read every receipt in the console or over the API, with viewer-level access. An error carries an identifier that looks up the exact receipt for that request, so a failure is a row you can point at rather than a support conversation.

The window. You can dispute any figure on any receipt in writing within 30 days, per section 9 of the terms. Where a request’s evidence is incomplete, it was already excluded from the fee, so a dispute is never an argument about a number we estimated.

The artifacts. A dispute is decided on records you hold or can export, not on our say-so: append-only receipts with both prices and the fee, an append-only audit journal, plans addressed by a content hash that either recomputes or does not, and confirmation reports whose statistics recompute bit for bit from the recorded seed. An edited report stops matching its own fingerprint.

The exit. Before or after leaving, you can export your evidence records and receipts and verify the export stands up without us; the bundle is checked row for row against the live API rather than summarized. Removing us from the serving path is one environment variable in your own code. A measurement company whose measurements only work while you are a customer would be arguing against itself.

If we still disagree. Raise it in writing and we have 30 days to resolve it before either side files, per section 20 of the terms. The aim of everything above is that a fee dispute never comes down to trust; if one ever does, the methodology has failed and we want to know that even more than you do.

This page is one half of a review. The other half, what we hold and what we cannot reach, is the security page.