The best local LLM for coding right now is Qwen3 Coder 30B A3B on a 24 GB card, Qwen2.5 Coder 14B at Q4 on 12–16 GB, and Qwen2.5 Coder 7B at Q4 on 8 GB. Every one of these runs fully offline, autocompletes and refactors real code, and costs nothing per token. If your GPU has less VRAM than the model needs, drop the quant one notch before you drop the model size. This guide matches models to VRAM brackets, compares the two coding families people actually argue about, and ends with runnable commands so you can be generating code locally in about five minutes.
Which local LLM should you use for coding on your GPU?
Pick by VRAM first, model second — the model only fits if the weights plus context fit in memory. This is the shortlist that keeps coming out on top in 2026 community benchmarks and day-to-day use:
| VRAM | Model | Quant | Weights on disk | Why it wins this bracket |
|---|---|---|---|---|
| 8 GB | Qwen2.5 Coder 7B Instruct | Q4_K_M | ~4.7 GB | Best tokens-per-second-to-quality ratio for autocomplete and small refactors |
| 12 GB | Qwen2.5 Coder 14B Instruct | Q4_K_M | ~9.0 GB | Whole-file edits fit in context; still 30+ tok/s on a 3060-class card |
| 16 GB | Qwen3 14B or gpt-oss-20b | Q4_K_M | ~9–12 GB | Better reasoning on ambiguous specs; 4090-class cards keep it fast |
| 24 GB | Qwen3 Coder 30B A3B | Q4_K_M | ~18.6 GB | MoE: only ~3B parameters active per token, so speed stays usable |
Two rules make this table work in practice:
- Leave 1–2 GB of VRAM headroom for the KV cache. A 14B model at Q4 plus an 8K-token context will not fit in a 10 GB budget — context counts toward the total.
- Go one quant down before you go a model size down. Q5/Q4 quants cost a few percent of quality; a 7B model instead of a 14B costs far more than that.
How much VRAM do you need for a local coding LLM?
The honest rule of thumb: VRAM needed ≈ quantised weights + 0.125 GB per 1K tokens of context at 8-bit KV cache. In plain numbers:
- 8 GB runs 7B–8B models at Q4 comfortably. Expect autocomplete-length answers, 4K–8K context.
- 12 GB is the sweet spot for 14B at Q4 — enough room for the model plus a realistic 8K–16K coding context.
- 16 GB opens 20B-class dense models and Qwen3 14B with longer context.
- 24 GB runs the Qwen3 Coder 30B A3B MoE at Q4, which is the closest thing to a cloud-quality coding model you can host yourself.
CPU-only? It works — llama.cpp will happily run a 7B Q4 on a laptop CPU at 5–10 tok/s — but treat it as a patience exercise, not a daily driver. For the tooling side of getting a runtime installed, the Ollama vs LM Studio comparison covers which local LLM tool to put underneath your models.
Qwen3 Coder vs DeepSeek: which is better for coding?
This is the matchup the autocomplete bars actually ask about, and the answer splits cleanly:
- Qwen3 Coder (30B A3B) is built for the edit loop: it follows instruction-format conventions for tool calling, produces consistent diffs, and — the decisive part — the MoE design means only ~3B parameters activate per token, so a single 24 GB card gets 40–60 tok/s. For IDE-style use, responsiveness is quality.
- DeepSeek V3/R1 line argues at a higher level: architecture decisions, tricky algorithms, multi-step reasoning. But the flagship is 600B+ parameters; locally you only run it heavily quantised on multi-GPU or Mac unified-memory rigs, and it writes prose about code faster than it writes code.
Local choice: Qwen3 Coder for the daily driver, DeepSeek only if you have the hardware to host it near-full-precision. At 8–16 GB the debate is moot — Qwen2.5/3 Coder models are the strongest thing that fits.
How do you run the best local LLM for coding with Ollama?
Five commands, from nothing to an OpenAI-compatible API your editor can use:
# 1. Install Ollama (Linux)
curl -fsSL https://ollama.com/install.sh | sh
# 2. Pull the model that fits your VRAM bracket (8 GB card shown)
ollama pull qwen2.5-coder:7b
# 3. Chat with it interactively
ollama run qwen2.5-coder:7b
# 4. Use it as an OpenAI-compatible API from any tool
curl http://localhost:11434/v1/chat/completions
-d '{
"model": "qwen2.5-coder:7b",
"messages": [{"role": "user", "content": "Refactor this fn to be async: add(a,b){return a+b}"}]
}'
# 5. Point tools that expect OPENAI_BASE_URL at it
export OPENAI_BASE_URL=http://localhost:11434/v1No API key, no rate limit, no per-token bill. On Linux the installer registers a systemd unit, so if the server ever misbehaves, journalctl tells you why — the journalctl cheat sheet has the exact filters for service debugging.
Is a local LLM good enough for real coding work?
Yes for the edit loop, no for the hard problems — and that split is exactly how you should use it. A 14B–30B local model handles refactors, boilerplate, test scaffolding, regex, and “explain this legacy function” faster than most cloud APIs round-trip. Where it loses to the big hosted models is long multi-file reasoning and obscure framework trivia — a 30B model simply knows less than a frontier model.
The workflow that works: keep a local model running for 90% of your keystrokes, and reach for a hosted frontier model only for the gnarly design questions. Your code never leaves the machine for routine work, which matters for client code, and the VRAM you already own quietly replaces a subscription.
— mrsaynothing