OpenAI prompt caching: check the model generation before the discount

Official documentation checked September 5, 2026 · OpenAI API scope · No API calls executed for this guide

OpenAI prompt caching can reduce the work needed to process repeated input, but the current rules are not identical across model generations. In particular, a budget based only on ordinary input and discounted reads can miss a cache-write category on newer models. Start with the actual model and response usage schema, then calculate the result.

This guide provides a provider-specific accounting method and an original test worksheet. It does not promise a hit rate or measured speedup. The Ollama calculator on this site is not an OpenAI billing calculator; it models different provider rules. Keep the examples below in a separate worksheet rather than borrowing Ollama's presets or included-credit assumptions.

Why older caching tutorials need a date and model label

The current OpenAI prompt-caching guide distinguishes GPT-5.6 and later from earlier models. Newer models support explicit and implicit breakpoints and a cache-write category; older models use implicit caching. Minimum cacheable lengths and lifetime behavior also differ. An old statement about every supported model can therefore be wrong even when it once described a useful default.

Before adjusting an application, record its model identifier, endpoint, SDK version and the date of the documentation used. Check whether the application sends raw conversation items or manages state through another mechanism. If a framework controls the request, inspect the supported configuration rather than adding a field that may never reach the provider. Treat migration as a small compatibility test, not a text replacement in a spreadsheet.

A current model example: reads and writes are distinct

The GPT-6 Astra model page currently lists standard rates of $10 per million input tokens, $1 for cached input, $12.50 for cache writes, and $50 for output. It also describes long-input and service-tier modifiers. The worksheet below assumes standard processing, a short enough request to avoid the long-input modifier, and no additional tool charges.

For contrast, the GPT-5.4 model page lists ordinary input, cached input and output at different rates. That is not an invitation to choose a model solely by its input price. It demonstrates why a pricing record needs a model and applicable conditions, and why a new model's accounting categories should not be projected onto every older setup.

Build a request-level ledger

In the Responses API, inspect usage.input_tokens and its input_tokens_details, including cached_tokens and, where applicable, cache_write_tokens. The caching guide's calculation treats ordinary input as total input minus reads minus writes. Keep output separate. See the official cost calculation.

Use a synthetic request with 12,000 total input tokens, of which 8,000 are read from cache and 3,000 are writes. Ordinary input is 1,000. With the Astra rates above, ordinary input costs $0.01, reads cost $0.008, and writes cost $0.0375. The input subtotal is $0.0555. If the response contains 500 billable output tokens at the stated output rate, output adds $0.025, making this simplified total $0.0805.

These counts are invented to demonstrate arithmetic, not copied from a customer request. Their purpose is to make double counting visible. Charging all 12,000 input tokens at the ordinary rate and then adding reads and writes would charge the same input more than once. Conversely, subtracting reads but forgetting the write premium would understate this example's input cost.

A reusable prefix needs a deliberate boundary

For newer models, an implicit breakpoint can include changing material after the shared instructions. An explicit breakpoint at the end of stable content can make that reusable section addressable when later content changes. Read the current boundary examples; a long shared beginning alone is not proof of a cache hit.

Imagine an application grading support replies against a stable rubric. The rubric and approved examples are shared, but each customer exchange differs. In your own request design, label the rubric version and identify exactly where the customer-specific content begins. A deployment should make this boundary reviewable. Do not move confidential content into a broadly shared template for the sake of reuse.

The current Responses reference documents prompt_cache_options for newer models, including mode and the currently supported 30m TTL. It also documents prompt_cache_key. These controls do different jobs; a routing key does not turn changing content into an identical prefix. Check your chosen endpoint and SDK before implementing them.

Judge the economics of reuse, not just the first request

Consider a hypothetical prefix whose ordinary processing cost is $0.08. At a 1.25 write multiplier, its first write costs $0.10. At a 0.1 read multiplier, each full reuse costs $0.008. Four total uses with one write and three reads cost $0.124 for the prefix, compared with $0.32 for four ordinary uses. That difference is conditional on the full reuse actually occurring.

Now assume all four requests write a different prefix and receive no reusable reads. The same prefix-sized material costs $0.40 instead. The comparison has reversed. These two synthetic cases show why a rollout needs actual read and write counts, and why one warm demonstration cannot justify a monthly savings forecast. Add changing input and output to both scenarios before estimating the complete task cost.

Keep the denominator honest when reporting improvement. A reduction in one prefix component is not an identical reduction in total spend. If output, tools or retries dominate your workload, optimizing a small input component may have a modest overall effect. Report both the component change and the full cost per successful task.

Validate with a controlled, small test

Prepare a non-sensitive fixture large enough for the chosen model's documented eligibility rules. Make one request, then a follow-up with the intended stable content unchanged. Save the usage categories, model, prompt version, timing and completion status. Compare what was read and written. Do not publish an assumed hit just because the second request returned quickly.

Next test the variations your real application will produce: a different customer question, a changed reference version, a realistic pause and a restart of the calling process. Keep each case separate. A useful result can be “this request shape did not reuse the prefix”; that finding prevents a false budget assumption and points to the exact configuration to investigate.

Timing should include the measure you care about. Time to first visible text differs from time to a complete usable result. Note cold versus warm conditions and avoid treating one timing sample as a stable benchmark. Review output quality alongside latency. Caching is an optimization of a workflow, not a substitute for determining whether that workflow solves the task.

Track drift without collecting private prompts

A compact operational log can record a non-sensitive prompt-version identifier, model, input categories, output, duration, error category and success status. You do not need raw customer text to compute token-weighted read share. Avoid putting secrets or personal email addresses into cache keys, public charts or troubleshooting screenshots.

Review model and price changes together with shifts in read share. A lower hit rate after deploying a new template may be expected at first; a persistent increase in writes deserves inspection. Separate genuine workload changes from regressions in request assembly. Preserve enough version information to compare like with like.

Caching configuration and data-retention policy are separate review items. Consult OpenAI's data-controls documentation for the selected features and account arrangement before processing sensitive material. This guide does not certify that an arbitrary retention setting meets your organization's obligations.

Frequently asked questions

Is OpenAI prompt caching always free to enable?

Do not confuse a configuration action with how tokens are billed. Newer models have a write category. Evaluate actual rates and reuse rather than relying on a slogan about automatic caching.

Can cached tokens exceed total input?

In the accounting scheme explained here, reads and writes partition the reported total input with ordinary input. If your transformed report violates that relationship, inspect schema mapping and aggregation before trusting the bill estimate.

Does this change ChatGPT subscription usage?

This article addresses API requests and API billing categories, not the usage allowance of a ChatGPT subscription. Keep the product and account type explicit when asking about limits.

Can I use the Anthropic formula unchanged?

No. Anthropic's ordinary input field and OpenAI's total input field have different meanings in these examples. Read the Anthropic caching guide for its separate categories. The site's methodology records why provider-specific assumptions matter.