ollama context window — the silent 4096 wall

The compressed answer: the default window is 4096 tokens, oversized pastes are trimmed without any client-visible warning, and the model still answers — fluently — about text it never read. Raise the window with num_ctx (request), OLLAMA_CONTEXT_LENGTH (server) or a Modelfile (model), confirm with ollama ps, and price the bill first: the window is RAM, roughly 144 KiB per token on qwen3:4b, linear until the box refuses.

Nov 07 ollama[638]: level=WARN source=llama_server.go:344
  msg="truncating input prompt" limit=2,050 prompt=15437 keep=4 new=2,050

# same second, on the client side:
POST /api/generate → 200 …  POST /api/generate → 200 — 2m40s later a fluent answer built from the kept tail — the client never sees the trim

You pasted the whole document. The answer came back smooth, confident, and about the wrong end of the document — not obviously wrong, just wrong about which lines it could even see. Nothing errored, nothing warned you. The only honest account of what happened lives in the server's journal, one WARN line you will never see unless you go looking. That is the context window doing exactly what it was configured to do: holding 4096 tokens and quietly dropping the head of the rest of your file.

What is ollama's default context window?

4096 tokens, and the server will print it — in the wrong places. Not at load time, not in the CLI banner, not in an error. ollama ps carries a CONTEXT column; the API twin is context_length in /api/ps:

$ curl -s localhost:11434/api/ps | jq '.models[0] | {name, context_length}'
{
  "name": "qwen3:4b",
  "context_length": 4096
}

# and the journal names the mechanism on a CPU box:
msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

The trap is the order of operations: context_length is readable only after a model is loaded — you meet it having already been burned. On the GPU-less bench box the default even logs its own reasoning: no VRAM found, so 4096. On a GPU host the default scales with VRAM instead — which is exactly why the number you got is never the number the guide assumed. A porthole sold as a picture window.

Why does ollama forget the middle of my document?

One 800-line document — every line carries a unique safe word — asked about its start and its tail at the default window, then about the same tail with the window raised to 16384. Every number below came off the bench this run:

The witness rule, stated once: if the paste matters, read prompt_eval_count. It sits in every /api/generate and /api/chat response, unit "tokens actually read". Send 15,437 tokens and see 2,050 evaluated — that gap is the truncation, measurable from the client the server never told.

How much of my prompt actually fits?

Less than the window says — and the reserve moves with the window. The WARN line's arithmetic is the quiet discovery of this bench: on the 4096 window the input cap measured 2,050 tokens — roughly half, the rest reserved for the reply the server assumes you'll want. At 16384 the same 15,438-token paste went in whole — no trim, no WARN — and spent 25 minutes in prompt evaluation before we cut it loose. So the reserve is not a number you memorize: it depends on the window and the reply-space the server keeps, and the WARN line is the only place it’s printed. The practical arithmetic for your own pastes: send the paste, then read prompt_eval_count — if it clamps under your token count, the head died; raise the window and resend.

How do I set the context window in ollama?

Three doors, all executed on the bench this run — none taken from docs. The per-request door is an option on the API call:

curl localhost:11434/api/generate -d '{
  "model": "qwen3:4b",
  "prompt": "...",
  "stream": false,
  "options": { "num_ctx": 16384 }
}'

verify: /api/ps flipped to context_length: 16384 and the same 15,438-token paste went in whole — no trim, no WARN, the window moved.

The standing door is the environment variable on the server — every model, every client, until you change it:

sudo systemctl edit ollama   # add:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=16384"
sudo systemctl daemon-reload && sudo systemctl restart ollama

verify: after the restart the server config dump named OLLAMA_CONTEXT_LENGTH:16384 and a fresh load printed context_length: 16384 with zero request options.

The third door bakes the window into a model — PARAMETER num_ctx in a Modelfile, ollama create once, every future run inherits it:

$ printf 'FROM qwen3:4b\nPARAMETER num_ctx 16384\n' > Modelfile.16k
$ ollama create qwen3-16k -f Modelfile.16k
$ ollama show qwen3-16k | grep -A2 Parameters

verify: the created model loaded with context_length: 16384 in /api/ps — the baked parameter took, no request options needed.

Precedence, measured: request option beats environment variable beats Modelfile beats the 4096 default — set one, check ollama ps, and the CONTEXT column tells you which one won. The CLI has no context flag at ask time at all; ollama run model "prompt" rides the server default, so terminal users get the env var or nothing.

Does a bigger context window cost memory?

Linearly, and the bill is the KV cache: every token of window reserves a slab of RAM whether you paste into it or not. Same box, load the model, raise the window, read free -m:

Two receipts from our own corpus belong beside this table. The kv cache quantization post (11-04) measured the crossover on this same model: past about 17k tokens, quantizing the cache itself to q8_0 pays for its accuracy cost — the next lever when you must run wide windows on a small box. The 128k lease piece (09-22) is this bill's long-form version: the window is a lease and this table is the payment plan, extended.

The context window checklist

• Read the window you actually got: ollama ps → CONTEXT column
• Halve it in your head: the input cap is ~half of the window, the rest is saved for the reply
• Pasting a document? One ask: "options": {"num_ctx": …}
• Terminal-only workflow? OLLAMA_CONTEXT_LENGTH on the service
• After the ask, compare prompt_eval_count to your paste size — a clamp means part of the document was never read
• Before raising it far, price the RAM: ~144 KiB of cache per token, linear

The unresolved thing, stated plainly: the full account of a truncation lives in the server's journal, one WARN line per event, and no client surface repeats it. The API returns 200. The CLI answers. prompt_eval_count is the one number that tells the truth from where you sit — until ollama surfaces the trim in the response itself, reading that number is the difference between a citation and an invention.