Ollama Not Using GPU? Fix It on Linux, Windows and WSL

The five-minute checklist

• ollama ps shows the PROCESSOR column
• If split or 100% CPU — run the journalctl line, read the inference compute output
• Find your state in the table below, apply the fix
• Restart the service and run ollama ps again to confirm

Work the causes in order — the fix is in the third column.

Symptom	Likely cause	Fix
100% CPU in ollama ps, NVIDIA card present	Driver too old for bundled CUDA	Update driver, reboot, restart service
100% CPU, AMD on Linux	Missing render/video group	usermod -aG render,video, re-login
100% CPU, unsupported AMD card	ROCm rejects the gfx target	HSA_OVERRIDE_GFX_VERSION=10.3.0
40%/60% CPU/GPU split	Model + context exceed VRAM	Smaller quant or shorter num_ctx
GPU worked yesterday, CPU today	Pinned OLLAMA_LLM_LIBRARY or stale driver	Find and remove the env var; update driver
GPU in native runs, CPU in Docker	Container launched without GPU flags	Recreate with --gpus=all (or AMD devices)

Check in this order: ollama ps for the state, server logs for the detection list, then the table. Nine times out of ten the log line already told you which row you're in.

How do I check if Ollama is actually using the GPU?

Two commands decide it.

# In one terminal: load a model
ollama run llama3.2 "hello"

# In another: see where it runs
ollama ps

NAME            ID          SIZE     PROCESSOR        UNTIL
llama3.2:latest a80c4de17cd9 3.3 GB   100% GPU         4 minutes from now

The PROCESSOR column has three states:

• 100% GPU — every layer offloaded. Done.
• 48%/52% CPU/GPU — partial offload. The GPU is working, but the model plus context didn't fit in VRAM. See the VRAM section below.
• 100% CPU — inference is on the CPU. The GPU was either not detected or turned off on purpose.

Then read the server log, which names the hardware Ollama actually found at startup:

journalctl -u ollama --no-pager | grep -i "inference compute"

On a healthy NVIDIA box you want a line like:

inference compute id=GPU-xxxx library=CUDA compute=8.9 driver=12.4 name=NVIDIA GeForce RTX 4070

No line at all, or one that ends with a CPU-only fallback message, and you've found your problem. The rest of this post is the five causes, most likely first.

Why does Ollama say "no compatible GPU discovered"?

On NVIDIA, the usual culprit is the driver, not CUDA. Ollama ships its own CUDA runtime libraries, so you don't need the CUDA toolkit — but the bundled runtime needs a driver new enough to talk to it. nvidia-smi working is not proof; it only proves a driver exists, not that it's recent enough.

nvidia-smi --query-gpu=driver_version --format=csv,noheader

# Debian/Ubuntu family
sudo apt install nvidia-driver-570
# Arch family
sudo pacman -S nvidia

After a driver update, restart the Ollama service so it re-detects devices — detection happens once at startup, not per request:

sudo systemctl restart ollama

verify: journalctl -u ollama --no-pager | grep -i "inference compute" → library=CUDA

If it still refuses, check that OLLAMA_LLM_LIBRARY isn't set anywhere — see the "after an update" section.

Why does Ollama use the GPU for only part of the model?

Partial offload is math, not a bug: model weights + KV cache must fit in VRAM.

Three ways to close the gap, cheapest first:

1. Smaller quant. Dropping from Q8 to Q4 halves the weight size with a modest quality cost. The trade-offs are laid out in GGUF quantization levels explained.
2. Shorter context. num_ctx dominates the cache size. 32K context on an 8 GB card means most layers stay on CPU.
3. Fewer GPU layers. The num_gpu option caps how many layers get offloaded. Setting it below the layer count guarantees a split — if someone set it in a Modelfile or API call, unset it.

One more trap: 100% GPU but still slow

A GPU that shows 100% GPU but runs slower than expected may be swapping over system RAM. Check ollama ps SIZE against your actual VRAM.

Why is Ollama not using my AMD GPU?

AMD on Linux needs three things, and all three are checkable:

1. ROCm in the build. The official install script bundles it. Confirm what the server detected:

journalctl -u ollama --no-pager | grep -iE "rocm|inference compute"

2. Group membership. The runtime needs /dev/kfd + /dev/dri — the render/video groups:

sudo usermod -aG render,video $USER
# log out and back in, then:
sudo systemctl restart ollama

The missing group is the most common "Ollama not using GPU on Ubuntu" post on every forum — the driver was never the problem.

3. A supported GPU — or an override. Unsupported RDNA2 consumer cards (gfx1031, gfx1032) fail detection even with a working ROCm stack. The standard workaround is pretending to be a supported card:

sudo systemctl edit ollama

[Service]
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"

Then sudo systemctl restart ollama. Unsupported but widely used — if it misbehaves, remove it. Rather own the backends than fight the auto-detect? That's llama.cpp vs Ollama.

verify: journalctl -u ollama --no-pager | grep -iE "rocm|inference compute" → your gfx target listed

On Windows, AMD support is narrower — check Ollama's supported-GPU list for your card before assuming the install is broken.

Why did Ollama stop using the GPU after an update?

Updates change one of three things, in this order of likelihood:

1. A pinned backend library. OLLAMA_LLM_LIBRARY forces one runner (cuda_v11, rocm, even cpu) and silently beats autodetection. Find it and remove it:

systemctl show ollama --property=Environment | grep -i llm_library
env | grep OLLAMA

Why the pin survives every update

It's a debugging switch that persists in shell profiles and service files long after the reason for setting it is forgotten.

2. The driver fell behind the runtime. Ollama upgrades bundle a newer CUDA runtime; your driver doesn't move until you move it. Same fix as the driver section above.
3. The service is a container and the flags are gone. A recreated Docker container without GPU flags is a CPU-only container. The NVIDIA invocation is:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

No --gpus=all, no GPU — Docker has no reason to be generous. AMD containers: same flags as the AMD section (/dev/kfd, /dev/dri, group adds).

verify: env | grep OLLAMA → empty (pinned library gone)

Does Ollama work in WSL2?

Yes — but the Windows driver does the work: install it on Windows, never a Linux driver inside the distro (an in-WSL driver breaks CUDA passthrough).

wsl --update   # from PowerShell
ls /dev/dxg    # inside WSL — must exist for GPU use

verify: ls /dev/dxg → device exists · ollama ps → 100% GPU

If the job is serving GGUF to more than one user at a time, Ollama is the wrong engine for the question — can vLLM run GGUF covers the plugin route and its GPU-only rule.

Rather skip the extra layer? The Windows build of Ollama runs natively and sees the GPU without WSL.

No — a CPU-only run works the same, just slower, and for small models on a fast CPU it can be perfectly usable. On Apple Silicon it's a non-issue: Metal uses unified memory automatically, and the only limit is how much RAM you're willing to share with the model. New to the jargon? Quant sizes and VRAM basics live in GGUF quantization levels explained.

GPU back in service? The next decision is the runtime itself — llama.cpp vs Ollama settles it from the public record. Every post in the cluster is indexed on the local-LLM hub.