Why is my Ollama so slow? Five usual suspects, ranked

Slow Ollama generations are almost always CPU offload: ollama ps prints the CPU/GPU split, and anything under 100% GPU means part of the model is running at CPU speed. After that, check cold reloads (the default keep-alive is five minutes), the context window's KV cost, the quant you pulled, and swap. Each has a one-line check below.

Is it running on the CPU?

The first check, every time:

$ ollama ps
NAME       SIZE     PROCESSOR        UNTIL
qwen:14b   12 GB    31%/69% CPU/GPU  4 minutes from now

100% GPU or nothing. A mixed line means the load-time balancer put the overflow layers on the CPU, and every token now bills at CPU rate. Yesterday's piece covered the offload cliff itself — here it is as the number-one suspect: a model that half-fits is slower than a smaller model that fits.

Is it reloading every time?

Ollama keeps a model resident for five minutes after your last request (the default keep-alive), then unloads it. The next request after a gap pays the full cold load again. Our bench ledger measured the cost on the 8-core box: the d3:2b rung took 11.8 s to cold-load before its first warm verdict landed in 0.8 s — a fourteen-fold difference between the first ask and the second.

# keep the model warm for an hour after each use
$ OLLAMA_KEEP_ALIVE=1h ollama serve
# or pin one model permanently
$ ollama run d3:2b --keepalive -1

If your workload talks to the model in bursts separated by coffee breaks, keep-alive is the whole bug.

Is the context window eating it?

The KV cache grows with the window, and attention cost grows with it too. The default window is small by design — this box's ollama ps shows CONTEXT 4096 — and every doubling books more VRAM and more per-token work. If you raised num_ctx for long documents and everything got slower, that is not a bug either: it is the bill for the window. Size the window to the task, not to the maximum the model supports.

Is the quant heavier than the task needs?

Q8_0 of a model is roughly twice the bytes of Q4_K_M, and those bytes are VRAM pressure first and decode work second. The d3 ledger again: the 2b rung held 3.7 GB resident and the 4b needed 7.1 GB — same family, same task, and on an 8 GB card the 4b is already flirting with the offload cliff while the 2b fits whole. Pull the smallest quant that answers well; treat every extra gigabyte as a step toward the CPU.

Is it swapping?

The failure under the failure: when the model misses VRAM and the CPU fallback misses RAM, the box starts swapping model weights to disk, and nothing recovers until you kill the process. The ledger's honest line for the 9b and 27b d3 rungs on that box was simply "won't load" — the honest answer at 8 GB of RAM. Watch for it with the usual tools (free -h, vmstat 1); the fix is the same ladder as always: smaller quant, smaller model.

The one-pass diagnosis

check	command	healthy answer
CPU split	ollama ps	100% GPU
Reload gap	time the second prompt after a pause	warm, within seconds
Context bill	your num_ctx vs the task	sized to the task
Quant weight	ollama list sizes	smallest that answers well
Swap	free -h under load	swap untouched

Read-from-record piece: the receipts come from our 2026-10-24 CPU bench ledger (cold load, warm verdicts, size rungs), the Ollama default keep-alive documented in its FAQ, and this box's own ollama ps output. No new bench today.

Slowness is almost never mysterious — it is the load-time balancer making a decision you didn't see. Make it show its work, then decide which cost you actually want to pay.