What does an AI gateway keep of your traffic?

Every gateway is in the path. Start from that.

A gateway serves your request by reading it. The prompt is in its memory, the completion streams back through its process, and no gateway architecture changes that. Any vendor in this category telling you your content never reaches them is describing a product that could not serve you. The honest sentence is the one on our own pages: your traffic goes through us. The differences begin the moment the response is delivered, because everything after that moment is a choice.

Three kinds of record can outlive a request. Content is the prompt and the completion themselves. Derived records are computed from content: semantic tags, categories, classifications, embeddings, quality labels. Operational records are about the request rather than its substance: model, tokens, latency, price, status. Every gateway keeps the third kind, because billing requires it. The category splits on the first two, and a vendor's documents will tell you which side it lives on if you read them in the right order: retention policy first, then the content licence in the terms, then the metadata definition.

The four retention postures, ranked by what reversing them takes

Recording by default. The gateway stores inputs and outputs unless you find the setting and turn it off. Ramp's Router, launched publicly in August 2026, documents this posture plainly: it records model inputs, outputs, and tool calls, retains recorded content for one year by default, and its FAQ states that opting out stops future recording but does not delete existing archives (their docs, August 2026). Its terms grant a worldwide, royalty-free, transferable licence to host, copy, modify, and store submitted content, for purposes that include improving the service and operating its business (their terms, August 2026). Documented plainly is the good version of this posture; the default is still the policy most keys run under, because a setting most users never open is not really a choice most users made.

Recording behind consent. Same storage, but off until you enable it or a feature that needs it. Better, because the archive that exists is one somebody asked for. The questions that remain are what the licence permits once content is stored, and what deletes it.

Zero retention as an account state. The gateway keeps no content, but the posture lives in configuration: a flag, a plan feature, an enterprise addendum. Real, and one settings migration or acquisition away from different. Watch for two qualifiers in this posture's fine print. One vendor's notice states that its content setting does not affect its collection or use of metadata, whose definition includes semantic tags, classifications, and categories computed from traffic, and that a gateway-side zero-data-retention setting does not by itself guarantee the model provider behind it applies zero retention (their notice, August 2026). Both qualifiers are honest, and both narrow the words zero retention considerably.

Absence by schema. The strongest posture is structural: no column exists that could hold content, so retention is not a setting anyone can flip and there is no archive an opt-out leaves behind. A schema can change too, but only through a migration, in public, with the policy pages that cite it changing in the same commit. The posture to want from any vendor is the one whose reversal would be loudest.

What Finest keeps, mechanically

On the Finest gateway, request and response bodies are used in memory to serve the request, to run any validator that decides whether to escalate, and to compute the receipt. Then they are gone. No table in this system has a column for gateway prompt or completion content, and there is no encrypted content store to keep it in, so this is a property of the schema rather than a retention setting somebody could change. Those sentences are quoted from our privacy policy, and the schema is what makes them true rather than aspirational.

The enforcement is layered so that a regression is loud. The journal that records how a request was served carries a database constraint that rejects any insert whose detail carries a prompt, messages, request, response, input, output, text, content, body, image, or document key. Receipts are append-only, with the runtime's permission to update or delete them revoked outright. Error events are rebuilt from an allowlist before they leave the process, so request bodies, local variables, and breadcrumbs are absent by construction rather than deleted by a rule an SDK upgrade could outrun. And the analytics vocabulary is bound to the privacy policy by a test: an event property that describes a prompt and is not named on the policy fails the build.

What does persist is named in one breath. The receipt: requested and served model, task class, decision, evidence label, token counts, both prices, savings, fee, and outcome. Operational logs: identifiers, routes, timings, and the arriving IP address, rolling off on a fixed schedule. And the operational metadata the terms define as records about a request rather than its substance, which is what serving decisions learn from. The model provider you addressed sees the request you addressed to it; every provider we can serve is listed publicly with the region it publishes, because that is the part of the path no gateway architecture removes.

Retention is half the question. The licence is the other half.

What a vendor stores matters less than what its terms let it do with what it stores, because the terms outlive today's architecture. Read the content clause of any gateway's terms and put the purposes side by side. At one end of the category, terms take a perpetual, irrevocable, sublicensable licence to inputs. In the middle sits the transferable licence scoped to purposes like improving the service and operating the business, which is the wording that turns a stored archive into a product asset. The clause tells you what the archive is for.

Finest's terms, section 8, are written to be diffed against those clauses. The licence you grant is limited to hosting, transmitting, processing, and displaying content for the sole purpose of providing the service to you; it is revocable, it terminates when the content is deleted or the agreement ends, and it is not sublicensable except to the provider you addressed and the sub-processors we publish. Explicitly, it does not permit us to train, fine-tune, distil, align, benchmark or evaluate any model on Customer Content, to sell or license it to anyone, whether or not anonymised, to build a competing product with it, or to publish it. There is no discount, tier, credit, or setting that buys a broader licence.

Two honest disclosures belong beside that. Aggregate, de-identified statistics about how model configurations perform are derived from operational metadata and from verification runs on open corpora, never from your content, and are never attributable to your workspace. And verification evidence in this release admits only public-reference, synthetic, and CI corpora; content-bearing production corpora are refused outright, and no configuration value enables them. That refusal is the protection, and we will not describe it as an enclave.

Five questions to ask any gateway, including us

Every answer below is checkable from documents the vendor already publishes, or from one support email. A vendor that cannot answer them crisply is telling you something too.

1. What does a fresh key record on day one?

Not what can be configured: what the default does before anyone opens a settings page. A recording default is the effective policy for most traffic. Then ask what deletes an archive that already exists.

2. What licence do the terms take in your content?

Find the clause and read the purposes. Improving the service and operating the business are the phrases that make stored prompts a product asset. Then check what the licence explicitly refuses, and whether any tier or setting widens it.

3. Does zero retention bind the model provider too?

A gateway-side setting governs the gateway. The provider that serves the request retains under its own terms, so ask for the provider list, each one's published region, and where each one's retention terms live.

4. What exactly persists per request?

Ask for the column list of the record that outlives a request. A vendor that stores content answers with a policy. A vendor that cannot store content answers with a schema, and the difference is the whole point.

5. What breaks if you start keeping more?

The strongest answer names a constraint: a database that rejects content-shaped writes, a build that fails when analytics vocabulary outgrows the privacy policy. Vigilance is a practice. A constraint is a fact.

Questions people ask

Do AI gateways store your prompts?

Many do, and by default. One major gateway launched in August 2026 recording model inputs, outputs, and tool calls, with one-year retention unless you opt out, and its docs state that opting out does not delete existing archives. Postures range from recording-by-default to schema-level absence, so read the retention policy and the terms' content licence before the model list.

Does Finest store prompt or completion content?

No. Request and response bodies are used in memory to serve the request and compute the receipt, then they are gone. No table in the system has a column for gateway prompt or completion content, so there is no retention setting to trust and no archive an opt-out would leave behind. What persists is the receipt: models, task class, decision, evidence label, token counts, prices, savings, fee, outcome.

Does Finest train models on my data?

No. The terms' content licence explicitly does not permit training, fine-tuning, distilling, aligning, benchmarking, or evaluating any model on Customer Content, selling or licensing it to anyone, whether or not anonymised, or publishing it, and no tier or setting buys a broader licence. Serving decisions do learn, from operational metadata defined as records about a request rather than its substance.

What is zero data retention (ZDR) for LLM traffic?

An arrangement where a provider or gateway keeps no copy of inputs and outputs after serving a request. Check two things before relying on the label: whether it is a schema fact or an account setting, and whether it binds the model provider behind the gateway, whose own retention terms govern what it receives. A gateway-side ZDR toggle does not by itself change the provider's behavior.

What should a gateway keep about a request?

The operational record that billing and audit require: which model was requested and which served, token counts, latency, prices, and outcome. On Finest that record is the receipt, it is append-only by database constraint, and it carries the counterfactual cost of the model you asked for, which is what makes a savings claim auditable rather than asserted.