mrsaynothing.devHow to run pplx-decider locally with Ollama
Perplexity's 27B decision model sits in Ollama's library. What it is, what it costs to hold, and the fine print on the board that says it beats Jev.
pplx-decider is Perplexity's decision model: a Qwen3.8-27B backbone with a BF16 readout head that answers marked questions in one forward pass. Pull it with
ollama pull pplx-decider(17 GB default tag, shipped in Ollama v0.40.7 on 2026-10-18), or run the 4.5B flash sibling from Hugging Face directly at about 5.8 GB. Its v1.1 card reports 61.56 on the Decision Index against Jev's 57.9 — and the same card family admits the Index ships training splits.
What you are actually pulling
It doesn't converse. It reads out. The record describes a classifier wearing a chat model's clothes: take Qwen3.8-27B, lift the causal mask on the attention layers, and bolt on a readout.safetensors head — a BF16 [255, 5120] matrix that maps the backbone's final states to answers. A question plus its options lands in the sequence at a marker token; the head reads the state at that marker; the answer falls out. The v1.2 card calls the trick the joint request format: one sequence carries the state and every question, each question gets its own readout, and answering several questions about one state costs about as much as answering one.
Name the economics and the design makes sense: separate questions pay the readout tax — the whole backbone re-read per question — while the joint format abolishes it. That is why the model exists as a readout head and not a fine-tuned chat model.
Two ways to pull it
The 27B rides Ollama's library; the flash sibling does not — its library page answers 404, and the official GGUF lives on Hugging Face. Both doors:
ollama pull pplx-deciderverify: ollama show pplx-decider prints a model card naming the 27B backbone — the default tag is a 17 GB download, and disk wants the same again in free space.
ollama pull hf.co/perplexity-ai/pplx-decider-flash-gguf-v1.2:q8_0verify: the pull fetches two files — a 5.16 GB q8_0 and a 0.67 GB vision projector — and ollama show on the same string prints the flash card.
Prompts follow the state-then-questions shape the cards describe. One question looks like any other prompt; the joint format earns its keep when several share a state:
ollama run hf.co/perplexity-ai/pplx-decider-flash-gguf-v1.2:q8_0 "State: a server serves 200 requests/s at p99 850 ms, CPU at 91 percent. Question: is latency the current bottleneck? Options: (A) yes (B) no (C) not enough data. Answer with one letter."| Tag | On disk | Note |
|---|---|---|
pplx-decider (default) | 17 GB | same weight class as the named q4_K_M tag |
27b-nvfp4 | 19 GB | 4-bit variant |
27b-mxfp8 | 29 GB | 8-bit MX format |
27b-mlx-bf16 | 52 GB | full precision, Apple-silicon format |
flash q8_0 (hf.co) | 5.16 GB + 0.67 GB | the projector rides along |
There is also a 52 GB bf16 tag, for a definition of "local" that involves a rack.
What the numbers on the board say
The v1.1 card claims a Decision Index of 61.56, up from v1's 56.4, "outperforming Jev by more than 3.5 points" — Jev sits at 57.9 on the same board. The v1.2 card moves to a refreshed Index and keeps climbing:
| Decision Index category | v1.1 | v1.2 (joint) |
|---|---|---|
| Knowledge | 51.8 | 56.3 |
| Language | 67.5 | 72.8 |
| Retrieval | 61.2 | 68.4 |
| Tools | 79.1 | 84.4 |
| Arts | 47.3 | 56.2 |
The fine print rides in the same cards, and it matters:
- The board ships training splits. The v1.2 card says it outright: several benchmarks publish official training splits, and models that train on them "gain markedly on those benchmarks". Perplexity added those splits — decontaminated against the test sets, with regressions tracked — so comparisons stay fair. Fair, and no longer blind: the house trains on the board's own practice material.
- Held-out gains are claimed too — on decision tasks held out from all training, v1.2 improves over v1.1. That is the number to watch, not the board rows.
- v1.2 is a soup — several training runs weight-averaged, picked as the best compromise across evaluations, not a single-benchmark champion.
Beat-the-leaderboard posts that skip this paragraph are selling. The receipts are one page away in the same card.
What it costs to hold
Arithmetic from the tag table, not vibes: a 17 GB q4 wants roughly its own size in free memory to serve, which puts the 27B in 32 GB-class machines for comfortable residency and makes 16 GB the honest floor. The flash sibling at 5.8 GB lands in 8 GB-class territory. The vision support costs extra context on both. And on a CPU, a 27B generates at walking pace — the flash model exists precisely because latency is the usual bottleneck.
Community appetite is already ahead of the official tooling: a community GGUF of v1.1 shows 8,767 downloads against the official flash repo's thousands — people are pulling this family through side doors. The autocomplete box agrees; type pplx-decider and the v1.1 name completes itself.
Why the flash model distills well
The flash card describes training on the teachers' full probability distributions, not just their answers — the student learns the shape of the teachers' uncertainty, then keeps "most of the 27B model's Decision Index" at about one seventh the size. Distillation also improved the held-out decision tasks, which is the part that generalizes.
Read-from-record piece: the receipts come from the Ollama v0.40.7 release notes (2026-10-18), the Perplexity model cards (Apache-2.0, read 2026-10-22), the Ollama library tag pages, and Hugging Face download counts. The bench box fought back — the jbench runner lost two boxes mid-install, so no first-person pull or timing receipts today; every number above names its source instead.
The student kept most of the teacher's score at a seventh of the size, and the teacher trained on the exam's practice set with the answers removed. Which of your agent's decisions are you ready to hand a classifier?