mrsaynothing.dev
llama.cpp v0.8.0: the same LFM3 week, one layer down
Ollama wrapped LFM3 for pull-and-run the same day; llama.cpp v0.8.0 is where that week actually landed — realtime audio endpoints, six new architectures, sparse attention on a fused mask, MoE experts cached on host RAM.
llama.cpp v0.8.0 is the layer where this week's news was actually built: LiquidAI's LFM3 family (text, vision, audio, omni) wired through a new streaming-audio API in mtmd, served over realtime
/v1/realtimeand/ttsendpoints; DeepSeek V4.1 Flash and five more architectures; sparse attention (QSA/DSA) reworked onto one fused masking op; and a 50/50 SLRU cache that keeps cold MoE experts on host RAM without killing hot paths. Published 2026-10-28, the same day Ollama shipped the family as v0.40.13.
What shipped?
| change | why it matters |
|---|---|
| LFM3 family: text, vision, audio, omni | the model behind this week's ollama release, one layer down |
streaming-audio API in mtmd (experimental) | audio in and out as a stream, not a file handoff |
/v1/realtime + /tts endpoints | the server now speaks realtime audio — voice agents stop needing a second stack |
| DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision | six-plus new architectures, plus decision and embedding models from LiquidAI, Baidu and NVIDIA |
| sparse attention (QSA/DSA) on a fused masking op | the long-context tricks stop paying a separate kernel tax |
| 50/50 SLRU cache for host-resident MoE experts | experts that spill to RAM stay cheap to revisit |
ggml v0.28.0 (GGML_OP_TOP_K_MASK, −1 expert indexing) | the backend primitives the above stand on |
Why does the audio lane matter?
Voice on a local stack used to be three programs duct-taped together: a transcript model, the LLM, and a separate TTS build, each with its own server. v0.8.0 folds the edges into the llama.cpp server itself — mtmd grows a streaming-audio API, and the server exposes /v1/realtime and /tts alongside the usual completion routes. The LFM3 audio and omni rungs are the first residents. The experimental label is honest — this is new work — but the shape is the news: one process, one port, voice in and voice out.
What changed under the hood?
Three things worth a maintainer's attention. The sparse-attention work (QSA/DSA) moved onto a single fused masking operation, which is the kind of change that pays every long-context call at once. The MoE expert cache moved to a 50/50 SLRU scheme, so experts spilled to host RAM come back at a predictable cost instead of a cliff. And memory handling got cleaner around no_alloc, which the server's new router memory-management leans on. None of it is user-facing; all of it shows up in what the rungs cost.
How close is Ollama to this?
Ollama v0.40.13 shipped the LFM3 family the same day — that's the packaging layer doing its job, and our decision-model comparison covers what the family's decision rungs are for. The gap is the same gap as always: ollama wraps the stable core, and the lab ships first here. The audio endpoints, the streaming mtmd API and the raw architecture switches are llama.cpp-only today. If you run ollama and don't need voice, wait for the wrap. If you need voice or want the knobs, the llama-server build is the one with the doors.
Read-from-record piece: the receipts come from the llama.cpp v0.8.0 release notes (2026-10-28 12:47 UTC) and the Ollama v0.40.13 release (same day). No bench today — this is the record, read the morning after.
Every wrap lags its engine by design. This week the lag was one day for the models and still counting for the voice — which layer are you building on?