Your hardware already runs assistants — the hard part is the matrix of quantization, runtimes and VRAM math nobody explains end-to-end. These field notes cover the whole path: pick a model, quantize it, choose a runtime, keep it on the GPU, and understand where your ram actually went.
- Best Local LLM for Coding: 8GB to 24GB VRAM Picks The best local LLM for coding by VRAM bracket: Qwen3 Coder vs DeepSeek at 8, 12, 16 and 24 GB, the right quant per card, plus a runnable Ollama setup.
- How to Run GGUF Models Locally: Ollama, llama.cpp & vLLM How to run GGUF models locally: one-line Ollama pulls, llama.cpp straight off a Hugging Face URL, and how to pick the right quant for your VRAM.
- GGUF Quantization: Which Level Should You Use? GGUF quantization levels compared: Q4_K_M vs Q8_0 perplexity, file size math, VRAM per level, and one rule for picking the right quant for your GPU.
- llama.cpp vs Ollama: Which Should You Run in 2026? llama.cpp vs Ollama: Ollama wraps llama.cpp for convenience, raw llama.cpp wins on speed and control. Benchmarks, GPU offload flags, and when each wins.
- Ollama vs LM Studio: Which Local LLM Tool Should You Use? Ollama vs LM Studio compared for real dev work: install, GPU use, speed and API serving — with runnable commands so you can pick the right tool today.
- Ollama Not Using GPU? Fix It on Linux, Windows and WSL Ollama ignoring your GPU and falling back to CPU? The five real causes — drivers, VRAM, pinned backends, ROCm, Docker flags — and the command that fixes each.
- 128K Context on a Desktop Is a Lie. The KV Cache Ate It. Model cards brag 128k context. The KV-cache arithmetic says your RAM pays for it up front — 16 GiB for an 8B model at full window. Do the math.