# The Request Compiler

> Serving every AI workload at its cheapest sufficient configuration: one model or a specialist team of them, with evidence measured on your traffic, and a floor that never goes away.

Finest Research · August 5, 2026 · canonical: https://finest.so/research

This is the plain-text edition of the article at https://finest.so/research. Same text, same tables, same
figures' findings, without the page. Cite the canonical URL.

## Abstract

Companies overpay for AI because they call frontier models at maximum settings for work that cheaper models, or cheaper *combinations* of models, perform at equal measured quality. Switching without evidence is terrifying, so nobody switches. We describe the Finest Request Compiler: a system that observes the anatomy of each workload without reading it semantically, measures cheaper configurations against the requested model as an immutable baseline, and executes selectively (substituting, cascading, ensembling, or decomposing requests) only where a confidence-bounded non-inferiority result has been earned on the customer’s own traffic. A database-enforced billing law (`debited = min(served, requested)`) makes the worst case literally "exactly what you asked for." On a real 82-document benchmark corpus, the method measured 48% lower cost and 26% lower latency with zero added hallucinations on ~90% of documents. Coverage is deliberately incomplete: abstention is a designed outcome, and we report it.

## Contents

1. [The overpayment problem](https://finest.so/research#overpayment)
2. [Workloads have anatomy](https://finest.so/research#anatomy)
3. [Execution: four strategies, one floor](https://finest.so/research#execution)
4. [Evidence before authority](https://finest.so/research#evidence)
5. [Case study: decomposing a document pipeline](https://finest.so/research#case-study)
6. [The economics of the attempt](https://finest.so/research#economics)
7. [The two-minute experience](https://finest.so/research#integration)
8. [Limitations](https://finest.so/research#limitations)
9. [Related work](https://finest.so/research#related-work)

## 1. The overpayment problem

The model wars produced a strange market: quality went flat while prices stayed vertical. Mid-tier
models now sit within a few points of the frontier on most workloads at a fraction of the price, and
frontier models ship with reasoning modes on by default, so a classification call pays output-token
prices to *wait* while the model thinks about a task that needed none of it. Claude Opus 5 costs $25
per million output tokens and Claude Fable 5 costs $50; Gemini 3.6 Flash costs $7.50, GPT-5.6 Luna
costs $1.20, and DeepSeek V4 Flash costs $0.28. The last increment of quality is an
order-of-magnitude step.

Engineering teams respond rationally: they pick a frontier model once, set max settings, and never
revisit it, because the alternative is re-running evaluations they don't have against models that
change monthly. The result is a market-wide overpayment that no one can see, because evidence about
models *on your workload* is exactly what doesn't exist. Everyone sells access to models. Nobody
sells proof about them.

Prices above are provider-official list prices (August 2026, Finest registry). Published benchmark
scores are directional orientation only: Finest admits no arm on benchmarks, only on evidence
measured against the customer's own workload.

## 2. Workloads have anatomy

The compiler's first move is not a model call. It is a deterministic observation: countable
structure, computed in microseconds, never embeddings or a serve-time classifier. Every request
carries an anatomy, and the anatomy decides what may be tried. This is the same lesson document
pipelines learned the hard way: born-digital versus scanned versus genomic-dense are knowable before
any expensive pass, and the safe direction for a classification error is toward the expensive model.

| Axis | Examples | What it decides |
| --- | --- | --- |
| Carrier / material | text, image, scan, table, code tree, structured file | parsers, perception systems, provenance rules |
| Composition | context-vs-directive ratio, enumerated sub-parts, history depth | whether splitting or distillation is applicable |
| Coupling | independent items · map/reduce · ordered · globally entangled | whether decomposition is safe at all |
| Repetition | same template, varying slots | per-template evidence, cache structure |
| Intent | task class, declared or inferred (precision-first, named abstains) | the class of the quality bar |
| Contract | schema, closed leaves, citations, streaming | what can be verified, when bytes may release |
| Consequence | disposable draft → user decision → regulated domain | the evidence and review bar |

Anatomy is stamped on every receipt as content-free facts, never prompt text, which produces a
census of real traffic before any routing is built for it. Each axis earns its build: below a
funding bar of measured demand, the observation stands alone as a trust fix and the routing waits.

## 3. Execution: four strategies, one floor

"Best of best" is not always model selection. The compiler admits four execution strategies per
request, plus a fifth, passthrough, which is a first-class success state. Every strategy shares one
property: any loud failure escalates to the requested model, and the customer is debited no more
than that model alone would have cost. The detour is our cost, by law.

- **Route.** Evidence picks the cheapest proven arm.
- **Cascade.** A cheap arm answers first under deterministic validators, escalating on loud failure.
- **Ensemble.** Two independent cheap arms must agree, or the requested model decides.
- **Plan.** A complex request is decomposed into specialist legs, each validated before the next
  spends anything.
- **Passthrough.** No safe plan yet: the requested model serves, and the fee is $0.00.

Why not simply send everything to the cheapest modern model? Because quality cliffs are
input-dependent and invisible on easy inputs. The measurements below come from a real extraction
benchmark run with production scoring rather than vibes, and they are the reason evidence gates
every routing decision.

| Model | Clean text | Hard input (rotated scan / dense) | The lesson |
| --- | --- | --- | --- |
| Haiku 4.5 | 100% recall | 18% recall · invented values (rotated scan) | perfect on text, unsafe alone on hard vision |
| GPT-5.4-mini | 100% recall | mis-typed all 75 dense-lab values as "condition" | the cliff appears only on dense inputs |
| GPT-5.4-nano | 100% recall | returned nothing (rotated scan) | silent give-up; undetectable without checks |
| Gemini 3.5-flash-class | 100% recall | 100% recall (both hard shapes) | the right fast vision arm, for this workload, this month |
| Opus 4.6 / 4.8 | 100% recall | 100% recall | the floor you're paying for, needed only where it's needed |

Every model is perfect on clean text. The failures only appear on hard inputs, and only measurement
finds them before production does.

## 4. Evidence before authority

A plan may influence a response only after it has climbed a ladder that is deliberately hard to
climb: observation, then shadow comparison against the requested model at our cost, then a frozen
plan confirmed once on a sealed, group-disjoint set, then a sticky canary with a circuit breaker,
then, and only then, active service within a declared eligibility envelope. Quality-affecting
changes (a model version bump, a prompt edit, a price epoch) invalidate authority until
re-confirmed. The model-wars churn that today forces every team to re-evaluate quarterly becomes our
scheduled maintenance.

The customer-visible states are: Observing → Testing privately → Verified for this workload →
Active. They are per workload, not per account: one workload may be Active while another is
Observing and a third is honestly idled. "No safe plan yet" is always available, and is a success
state rather than a failure: the requested model serves and the fee is $0.00.

The verifier hierarchy is honest about its limits. Deterministic checks (schema, conservation,
provenance) authorize only the property they actually decide. Calibrated learned checks carry
measured false-pass behavior. Blinded cross-family judges are supporting evidence, never promotion
authority. And the requested model itself is the measured baseline: *never ground truth merely
because it was requested.* Where nothing machine-checkable applies, the system abstains, the receipt
says so, and no fee is earned.

## 5. Case study: decomposing a document pipeline

A health-records pipeline began, as most do, by sending every document to the strongest available
model. Its operators then decomposed the job into typed capabilities (OCR, extraction, verification,
classification), measured twelve models against four document shapes with the production scorers,
and re-assigned each leg to the cheapest sufficient executor while keeping the verification gate on
the strongest model. The routing engine shipped dark, behind flags, with residency asserted at
construction: a configuration that would route data to an ineligible endpoint refuses to build.

| Pipeline leg | Before | After (routed) | Why it's safe |
| --- | --- | --- | --- |
| OCR · scanned pages | Opus 4.6 · 23.6s · $$$ | Gemini 3.5-flash · 3.2× faster | transcription only; grounding + verify backstop; consensus on dense/rotated |
| Extract · clean born-digital | Opus 4.6 · 22.8s | Haiku 4.5 · 10.6s · 100% parity | fail-closed to Opus on truncation/error; doc-shape override upward |
| Extract · genomic / dense | Opus 4.6 | Opus 4.6 · unchanged | deterministic classifier measured the recall gap (86-page genomic report: −37% on Haiku) and routes up |
| Verify · every fact, always | Opus 4.6 | Opus 4.6 · never routed | the trust gate; a routed model can only degrade a fact to review |
| Classify · image gallery | Opus 4.6 | Gemini flash-class · cheap | off the ready-path; latency-insensitive |

Measured aggregate on the 82-document real corpus: **48% cheaper, 26% faster, zero added
hallucinations on ~90% of documents.** These are benchmark aggregates, honestly scoped: not a
promise about your workload, which gets its own measurement.

Two details matter more than the headline numbers. First, the failures were found by measurement,
not prevented by intuition: an 86-page genomic report lost 37% of its facts on the cheap extractor,
so a deterministic doc-shape classifier now routes those documents upward, a classifier that can
raise cost but never weaken the gate. Second, the one error class a verifier cannot catch, an OCR
misread that grounds against its own transcription, was closed not with a stronger single model but
with *independence*: a second OCR read whose disagreements surface to review. Consensus is
disagreement evidence, and it is the only mechanism that works there.

## 6. The economics of the attempt

Two questions kill any system like this if they aren't answered in numbers: who pays for failed
cheap attempts, and what happens to latency. Our answers are structural.

On money: escalations are capped per request (retries are zero by design), every loss is recorded as
signed negative savings on an auditable ledger, and routed cells auto-suspend when their measured
escalation rate crosses a ceiling, long before the portfolio math turns negative. At provider list
prices the portfolio stays profitable until roughly 37–94% escalation depending on the
champion-to-cheap price gap; graduated cells measure ≤8%. **This is our money. We pay for every
failed cheap attempt, which is why we cannot afford to be wrong.**

On latency: the median improves, because cheap models answer faster than frontier; the tail is
escalations, bounded at roughly champion + cheap, measured per cell and controllable per request
with one header. Multi-leg plans ship to relaxed-latency workloads first.

## 7. The two-minute experience

None of this should be the customer's job. The integration is a key swap; the observation and shadow
evidence are free and automatic; the only question is a one-tap activation (provider boundary,
posture, latency), asked after the measured report has already shown what overpayment looks like on
their own requests.

```sh
# the entire integration
ANTHROPIC_BASE_URL=https://api.finest.so/v1
ANTHROPIC_API_KEY=fk_your_key
```

Everything else is harvested, not asked: schemas from the codebase, shape from traffic, constraints
from a one-time boundary declaration. New model releases are re-measured against the customer's
workloads automatically. Companies should focus on their core business, not on tracking the model
wars. That is the product.

## 8. Limitations

- **Coverage is deliberately incomplete.** Tightly coupled reasoning, holistic creative work, and requests with no verifiable joint are not decomposable by this method; the system abstains, and abstention rates are published per workload, not averaged away.

- **Correlated errors escape comparison.** Two systems can share an omission. The requested model is a baseline, not truth; consensus across provider families and deterministic conservation checks narrow but do not eliminate this class.

- **Synthetic corpora are not production.** Minted corpora give free, exact labels at the decision boundary; they are guarded by real-distribution holdouts, and arms that regress there are re-minted before anything publishes.

- **The counterfactual can lie if estimated.** Pricing the requested model at a cheap arm’s token usage can invent savings (verbosity differs per model). Graph savings stay estimated and fee-free until an observed or reconciled baseline exists.

- **Benchmark scores here are directional.** Prices are provider-official; quality scores are public-benchmark orientation. Finest admits nothing on benchmarks; admissions happen on your workload’s evidence, and your workloads will differ.

## 9. Related work

Cost-aware routing is established: FrugalGPT [1], RouteLLM [2], and AutoMix [3] built cascades and
learned routers; compound-AI framing [4] and Anthropic's workflow taxonomy [5] describe the design
space; decomposed prompting [6] shows modular subtask routing while newer work warns that incorrect
decomposition and synthesis can degrade end-to-end reasoning [7]; Mixture-of-Agents [8] demonstrates
portfolio gains at portfolio cost; DSPy and successors [9] optimize prompts when a metric and
examples exist; semantic uncertainty [10] supports calibrated abstention; and judge-evaluation work
[11] is why no uncalibrated judge sits on our serving path. Agent-security work [12] informs the
trusted-control/untrusted-data separation in the plan runtime.

What these leave open, and what we claim as the product, is the contract around the machinery: an
immutable passthrough baseline, evidence-bound activation, per-request receipts, a database-enforced
billing law, and abstention as a reported first-class outcome.

## References

1. FrugalGPT: how to use large language models while reducing cost and improving performance. https://arxiv.org/abs/2305.05176
2. RouteLLM: learning to route LLMs with preference models. https://arxiv.org/abs/2406.18665
3. AutoMix: automatically mixing language models. https://proceedings.neurips.cc/paper_files/paper/2024/hash/ecda225cb187b40ea8edc1f46b03ffda-Abstract-Conference.html
4. Berkeley AI Research: Compound AI Systems, the shift from models to compound systems. https://bair.berkeley.edu/blog/2024/02/18/compound-ai-systems/
5. Anthropic: Building effective agents, workflows vs. agents; routing, chaining, parallelism, evaluator loops. https://www.anthropic.com/engineering/building-effective-agents
6. Decomposed Prompting: a modular approach for solving complex tasks. https://arxiv.org/abs/2210.02406
7. Reasoning can hurt: decomposition, subtask, and synthesis failures as distinct error classes. https://arxiv.org/abs/2505.24225
8. Mixture-of-Agents enhances large language model capabilities. https://arxiv.org/abs/2406.04692
9. DSPy: compiling declarative language model calls into self-improving pipelines. https://arxiv.org/abs/2310.03714
10. Semantic entropy: detecting confabulations with semantic uncertainty. https://www.nature.com/articles/s41586-024-07421-0
11. JudgeBench: a benchmark for evaluating LLM-based judges. https://arxiv.org/abs/2410.12784
12. AgentDojo: a dynamic environment to evaluate prompt-injection attacks and defenses for LLM agents. https://arxiv.org/abs/2406.13352

## Cite this work

```bibtex
@misc{finest-request-compiler-2026,
  author = {Finest Research},
  title  = {The Request Compiler: Serving every AI workload at its
            cheapest sufficient configuration},
  year   = {2026},
  month  = aug,
  url    = {https://finest.so/research}
}
```

---

prices: provider list, verified august 2026 (Finest registry) · benchmark scores directional · the case-study aggregates (48% / 26% / ~90%) are benchmark results on one real 82-document corpus, not production-outcome promises: your workloads are measured, not assumed · "no safe plan yet" is a designed state: coverage and precision are reported separately, never averaged