Prompt Caching and Batch Pricing, Explained

Cloud AI pricing has two discount mechanisms that most people never use, partly because the documentation explains the mechanism rather than the situation it fits.

Prompt caching can cut costs on repeated content dramatically. Batch processing typically halves the rate for work that can wait. Claude Opus 5’s pricing, for example, advertises up to 90 percent savings with caching and 50 percent with batching, and similar structures exist across providers.

Here is what each actually does, and the important caveat that neither applies to the thing you probably do most.

Prompt Caching

Every request you send includes the same preamble: your system instruction, your style guidelines, your examples, whatever context defines the task. The model processes all of it before it gets to the part that changed.

Prompt caching stores the processed form of that stable prefix. On subsequent requests with the same prefix, the provider reuses the cached computation instead of redoing it, and charges a fraction of the normal input rate for those tokens.

Where it pays off: long stable instructions, few-shot examples reused across requests, a document you are asking many questions about, or a codebase loaded once and queried repeatedly. The longer the shared prefix and the more requests share it, the larger the saving.

Where it does not: short instructions, or a prefix that changes between requests. If your system prompt is two hundred tokens, caching it saves a rounding error.

The detail that catches people: caching is prefix-based. The cached portion must be byte-identical from the start of the request. Putting a timestamp, a request ID, or the user’s text at the beginning invalidates the cache for everything after it. Stable content first, variable content last.

Batch Processing

Batch APIs accept a collection of requests, process them when the provider has spare capacity, and return results within a stated window, typically up to twenty-four hours. In exchange the rate is usually about half.

Where it pays off: anything not interactive. Classifying a backlog, generating summaries overnight, processing a document archive, evaluating a dataset, running a test suite against a model.

Where it does not: anything where a person is waiting. Which is most writing assistance.

The detail that catches people: the window is a maximum, not a promise of speed. Sometimes results arrive in minutes; sometimes they take hours. If your workflow depends on knowing when, batch is the wrong mechanism.

Neither Helps The Interactive Rewrite

Here is the honest limitation, and it is the reason this article exists.

You have written an awkward email. You want it in a professional register. You are waiting for the answer.

Batch is out by definition, because you are waiting.

Caching helps only marginally, because a rewrite instruction is short. Caching a two-hundred-token instruction saves a fraction of a cent on a request that already cost a fraction of a cent. It does nothing for latency, since you still wait for the output tokens to generate.

Which points at the real economics of interactive AI: at 2026 prices, cost is not the interesting variable for a person doing a dozen rewrites a day. That is single-digit dollars a month even on a frontier model. The interesting variables are latency and where the text goes. See open weights vs cloud API costs.

Where The Latency Actually Goes

Since caching does not fix waiting, it is worth knowing what does.

Round trip. Network latency to the provider and back. Tens to hundreds of milliseconds, unavoidable with a hosted model.

Queueing. Under load, requests wait. Invisible and variable.

Deliberation. If a reasoning model is thinking before answering, that is often most of the time on a short task. Turn the effort setting down. See thinking models versus instant models for rewriting.

Generation. Proportional to output length, which is why a length constraint in your instruction improves speed as well as quality.

Your own context switch. The largest component and the one nobody measures. Switching to a browser, waiting for the page, pasting, reading, copying, switching back: forty seconds and a broken train of thought. See the context switching productivity killer.

A local model removes the first two entirely and the last one almost completely, because the tool appears over whatever you were doing. That is why a small local model feels dramatically faster than a frontier cloud model even when raw generation speed is comparable.

When To Bother With Either Mechanism

Use prompt caching if you are building something that sends many requests sharing a long prefix. Structure your requests so the stable part comes first, and check the provider’s documentation for minimum cacheable length, which is usually not trivial.

Use batch processing if you have a queue of work and nobody waiting. It is close to free money for that shape of task.

Ignore both if you are a person using AI interactively. Spend your attention on the instruction quality and on routing tasks to appropriately sized models, which affect your experience far more than pricing mechanisms do.

The Instruction Is Still The Best Optimization

Worth restating, because it beats every pricing feature on every axis.

A precise instruction produces shorter output, which costs less and generates faster. It also produces better output, and it prevents the failure mode where a model expands three lines into four paragraphs:

Rewrite this as a professional work email. Corporate register, complete sentences, no contractions. Lead with the ask and the deadline. Keep every name, date, figure, and commitment exactly as written. Do not add enthusiasm or context that is not in the original. Keep the result no longer than the input. Return only the rewritten text.

That length constraint is a cost optimization, a latency optimization, and a quality improvement simultaneously. No caching configuration does that.

Wrivio Contexts store instructions in this form per situation, so the precision is applied automatically rather than depending on you typing it out at the end of a long day.

Common Questions

How much does prompt caching actually save?

Depends entirely on prefix length and reuse rate. Advertised figures assume a long stable prefix reused frequently. With a short instruction, the saving is negligible.

Does caching mean the provider stores my prompts?

It stores processed prefix data for a limited window to serve your subsequent requests. Read the provider’s documentation on cache retention and scope if that matters to you, and treat it as another retention question. See what zero data retention actually means.

Can I use batch for anything user-facing?

Only where the user does not expect an immediate answer: an overnight report, a queued analysis. Not for anything they are watching a spinner for.

Is local inference cheaper than a cached cloud call?

Per request, yes, since the marginal cost is electricity. The comparison stopped mattering much at 2026 prices; latency and data handling are the reasons to choose local.

Download Wrivio for Windows to remove the round trip entirely for the rewrites you are actually waiting on.