How to cut LLM API costs
Last updated 2026-08-19
Cut LLM costs in this order: stop sending easy work to frontier models, since published prices differ by multiples for work a smaller model does at quality; use prompt caching, which prices repeated context at up to 90% off on major providers; move latency-tolerant jobs to batch endpoints at half price; cap output length, because output tokens cost several times input tokens; put hard spend caps on every key; and measure per request, because a saving nobody can verify does not survive its first quality scare.
- 3.4×: 3.4× separates the upper-quartile endpoint (Claude Sonnet 4.6) from the lower-quartile one (Grok 4.20 (reasoning)) on the same 2,000-in / 400-out task, across 27 verified standard-tier endpoints at the price epochs in force now. The middle half of the catalogue rather than its extremes: dividing the most expensive frontier model by the cheapest small one is arithmetically true, describes a swap nobody makes, and reads as a typo. Published list prices, not a measurement of quality: whether the cheaper one does your job is the question the rest of this page is about.
- 5×: 5× is the median ratio of output-token price to input-token price across the same 27 endpoints. It is why an uncapped response is the most expensive thing a route can do by accident.
- 50%: Every provider publishing batch terms in the registry discounts by 50% (3 providers). Batch is a migration, not a routing flip, the caller has to submit, poll and correlate, which is why Finest generates it as a reviewable diff rather than switching it on for you.
Provider list prices read from the endpoint registry Finest prices with, recomputed on every render.
Lever 1: stop over-modeling. The largest saving on most bills
Production traffic is mixed: extraction, classification, reformatting, and routing glue share a bill with genuinely hard reasoning. Pinning one frontier model prices all of it identically, and the spread between frontier and efficient models on published rate cards is a multiple, not a percentage. Moving the traffic that clears a quality bar onto the model that clears it is the single largest lever, which is why it is Finest's entire product: measure which configurations hold quality per task class, serve those, and serve your requested model verbatim everywhere else.
Do this with evidence or not at all, because quality cliffs are input-dependent and invisible on easy inputs. In the measurements behind The Request Compiler, Finest's published method, every candidate model scored perfectly on clean text; the failures appeared only on hard inputs, where one efficient model dropped to 18% recall on a rotated scan and another silently mistyped all 75 values in a dense lab panel. A quality bar you did not pre-register will bend when the savings look good, and a substitution you cannot defend will be rolled back at the first complaint, taking the savings with it.
Right-sizing is also not only model selection. The compiler admits five execution strategies per request: serve the cheapest arm that holds the published bar, cascade from a cheap arm under validators, ensemble two cheap arms that must agree, plan a complex request into validated specialist legs, and passthrough, where no safe plan exists yet, your requested model serves, and the fee is zero. On the paper's real 82-document corpus, that method measured 48% lower cost and 26% lower latency with zero added hallucinations on about 90% of documents: corpus aggregates, honestly scoped, not a promise about your workload.
Lever 2: prompt caching. Up to 90% off repeated context
Most applications resend the same system prompt, schema, and reference documents on every call. Providers now price that repetition separately: cached input on Anthropic bills at a tenth of the base input rate, and OpenAI applies an automatic 50% discount to repeated prefixes. The engineering is mostly ordering: keep stable content first, volatile content last, and the discount follows.
Lever 3: batch endpoints. Half price for anything that can wait
The major providers publish 50% discounts for asynchronous batch processing with completion windows measured in hours. Backfills, evaluations, enrichment, nightly pipelines: if nobody is watching a spinner, it belongs on the batch endpoint. This is the easiest large discount in the category because it requires no quality judgment at all, only patience.
Lever 4: output discipline. The expensive tokens are the ones you generate
Output tokens are priced at a multiple of input tokens on current rate cards. Verbose answers, restated context, and unbounded max_tokens settings buy the priciest tokens on the invoice. Set explicit output budgets, ask for the shape you need rather than prose around it, and the bill falls with no model change at all.
Lever 5: hard caps. A runaway loop should die at a number you chose
One retry loop calling a frontier model overnight erases a quarter of careful optimization. Per-key limits on requests, tokens, and spend convert that risk into a bounded, alertable event. Finest keys carry rate and spend caps as a first-class feature, and the discipline generalizes: no key without a ceiling.
Lever 6: per-request proof. Savings that are not receipted get reversed
The failure mode of cost work is not technical. A month after the optimization, a quality incident appears, nobody can show which requests were affected or what the substitution actually saved, and the safe decision is to roll everything back. A receipt per request, naming the model served, the evidence that authorized it, and the counterfactual cost against your requested model, is what makes savings permanent. On Finest this is automatic, and it is what the fee is computed from: 25% of proven savings, nothing when there are none.
Questions people ask
- What is the single biggest way to reduce LLM API costs?
- Right-sizing models per task. Published prices differ by multiples between frontier and efficient models, and most production traffic does not need the frontier. Caching and batch discounts stack on top.
- How much does prompt caching save?
- Cached input reads are billed at up to 90% off base input rates on Anthropic and 50% off on OpenAI, per their published pricing. The saving applies to the repeated prefix of your prompts.
- Can I cut costs without any quality risk?
- Caching, batch endpoints, output budgets, and spend caps carry no quality risk. Model right-sizing does, which is why it should happen only behind measured, pre-registered quality bars with escalation, the way Finest serves it.
- Is there published research behind this method?
- Yes. The Request Compiler (Finest Research, August 2026) describes the full method: workload anatomy, the five execution strategies, the evidence ladder a configuration climbs before it may serve, and an 82-document case study that measured 48% lower cost and 26% lower latency with zero added hallucinations on about 90% of documents. It is at finest.so/research, including a limitations section.
- How does Finest charge for this?
- Tokens at the host's published rate with no markup. Finest's fee is 25% of the saving it proves on a request against your requested model. No proven saving, no fee.
In 2 minutes, start cutting your API spend without sacrificing quality. Free if you don’t save money.
No model markup. You pay the host’s rate. 25% of what it proves it saved on a request. No saving, no fee.