What is a context window in an LLM, and what does it cost?
Understand context-window capacity versus token consumption, reserve room for output, and compare repeated requests without treating the advertised limit as a bill.
A context window is the bounded working space a language model can use for a request. It is not a monthly token allowance, a measure of permanent knowledge, or a price. The practical question is whether the information needed for the next useful response fits, with enough room left for that response.
A larger window can make a task feasible, but selecting a model with a large advertised window does not mean every request uses the entire window. Capacity and consumption are different quantities. This guide builds that distinction with a synthetic document-review example. Documentation checked September 5, 2026; model limits remain provider- and version-specific.
Four numbers that should not share one label
| Number | Question it answers | What it does not answer |
|---|---|---|
| Context capacity | How much can this request accommodate? | How many tokens you used this month |
| Actual request input | How much material entered this request? | How much output will be useful |
| Output limit | How much generation is permitted? | How many tokens will actually be generated |
| Billing-period consumption | What work accumulated across requests? | Whether the next request fits |
These distinctions prevent two opposite errors. Someone can have plenty of unused account allowance and still send an oversized request. Someone else can make thousands of small requests, none close to the context limit, and accumulate substantial usage. A billing cap controls spending; a context check controls the shape of an individual request.
Count the request, not just the last message
The Claude API context guide explains that context includes request content such as instructions, history, tool definitions and results, plus generated output. Cached input still occupies context. Other providers may expose different accounting details, so check their exact request contract rather than transferring a model's advertised limit across services.
For your own planning, separate essential instructions, current evidence, historical discussion, tool overhead, and output reserve. This inventory is more useful than saying “my prompt is only one sentence.” That sentence may be accompanied by many files or an extensive conversation. A request can grow without the visible user message becoming any longer.
A document-review example
Suppose a fictional deployment has a usable 32,000-token window. You plan to send 2,000 tokens of instructions, 18,000 tokens of source material, and 5,000 tokens of prior discussion. That totals 25,000 input tokens, leaving a theoretical 7,000-token difference. Reserving 4,000 for the answer leaves 3,000 for other included material and estimation error. These are illustrative allocations, not the stated limits of a real model.
If a tool then appends 6,000 tokens of search results, the plan no longer has the same margin. You could remove obsolete discussion, select fewer relevant results, split the source material into independently answerable sections, or choose a verified larger-capacity deployment. Each option changes the information available to the model, so evaluate answer quality rather than measuring success only by whether the API accepts the request.
An output reserve is also a task decision. A structured extraction with ten short fields needs different room from an annotated review of twenty clauses. Asking for an unnecessarily long answer can consume resources without making the decision better. Conversely, starving the response budget may cut off the one explanation needed to verify a conclusion.
Repeated context changes consumption
Now imagine five independent review requests that each submit the same 10,000-token background packet and generate 1,000 tokens. Ignoring all other overhead, the total is 50,000 input tokens and 5,000 output tokens. The packet is still only 10,000 tokens in size, but it has participated in five requests. A capacity number cannot replace that accumulated usage ledger.
For synthetic rates of $2 per million input tokens and $6 per million output tokens, uncached consumption would be $0.10 plus $0.03, or $0.13. A supported cache can alter the input price, but the cost of caching depends on the provider's rules and actual reported usage. Do not mark repeated text as a cache hit just because the text looks identical to you.
This example also explains why a conversation that gradually grows can become expensive even when no one pastes a giant document. Each new action may bring forward an accumulated history. The Ollama API usage guide explains how to use completed response metrics to measure that work rather than guessing from transcript length.
What should stay in the working context?
Start with the decision you want the next response to support. If the task is to identify a failing validation branch, include the relevant function, the failing input, the observed output, and the contract it should satisfy. The entire product roadmap and a year of unrelated logs probably do not help that decision. Less information is useful only when the remaining information is sufficient.
Preserve evidence that a reader can independently check: filenames, precise section references, concise test results, and unresolved assumptions. Replace repetitive narrative with a small state summary. Keep the original source available outside the prompt so a disputed detail can be fetched again. Summaries reduce context load, but they should not become untraceable substitutes for evidence.
Our proposed comparison method uses two prompts on the same task: one broad collection and one curated packet. Evaluate correctness, omissions, elapsed time, and measured usage. This is an experiment, not a promise that smaller input always produces better answers. A broad task genuinely requiring many cross-document relationships may justify more context.
Local memory and hosted context are different constraints
In a local deployment, configured context can affect the memory required to run the model. Ollama's context-length documentation explains the memory tradeoff and shows ollama ps for inspecting allocation and offloading. The setting you intended to apply is not as strong as evidence from the process actually serving the request.
A successful short response is not a stress test of the maximum configuration. Test a realistic workload before committing your computer to an always-on service. Leave resources for the operating system and other applications, then increase only the dimension the task needs. Our Docker planning guide separates local resource costs from hosted billing.
How this relates to the site calculator
The monthly Ollama calculator estimates consumption from average input and output assumptions, then compares applicable subscription cash outlay. Enter realistic per-request usage, not a model's maximum context window unless each request actually approaches it. Treat the calculator as a forecast based on your measurements, not a validator that a prompt will fit.
If you have separate workloads, calculate them separately before aggregation: short classification, longer document review, and multi-step coding need not share the same token average. Keep model/provider differences explicit. A neat single average can conceal a small group of large requests that dominate both capacity risk and cost.
Frequently asked questions
Does a bigger context window mean a smarter model?
It means a different capacity constraint, not a complete quality ranking. Compare the model on your actual task, including whether it uses the supplied evidence correctly. Capacity matters when information must fit; it does not by itself establish reasoning reliability.
Do cached tokens stop counting toward context?
No for the Claude API behavior discussed here: the documentation distinguishes the price treatment from the occupied space. Apply the corresponding provider's documented rules when working elsewhere. A discount should not be interpreted as extra capacity.
Can I increase capacity by changing an environment variable?
A variable may change a local configuration or a client's assumption, but it cannot override a hosted server's real limit. Verify what the serving deployment accepts and what memory the local process actually has. For agent-specific configuration, see the Claude Code guide.
What if the request is already failing?
Use the context-window-exceeded diagnostic steps. First distinguish an oversized input from a response that ran out of room, a rate limit, and an account allowance problem. Those cases require different fixes.