ollama speech to text — the ear is the picture door

Last tested 2026-11-02 on ollama v0.40.13 (current latest, tagged 2026-10-28), model lfm3-audio:2b, a throwaway two-thread Arch VM with 8 GB RAM. Every number below is from that run; the full request log lives in our bench receipts.

Does ollama do speech to text?

The model card says yes: LFM3-Audio-2B takes text or audio at 16 kHz, transcribes across 18 languages, and answers in text — 2 billion parameters, 32K context, licensed under the LFM Open License. Ollama lists it as lfm3-audio:2b. The pull is 1.7 GB; resident, it sits at 1.6 GB and runs 100% CPU with a 4096 context on our two-thread box.

ollama pull lfm3-audio:2b

verify: ollama list shows lfm3-audio:2b · 3327c1a06755 · 1.7 GB — last tested 2026-11-02.

The card is not the question. The question is which API door actually carries sound into it — and that answer is not written anywhere official. So we rang all three doors with the same file: a 16 kHz mono wav of espeak-ng saying "The quick brown fox jumps over the lazy dog." (3.65 seconds, 16-bit mono, nothing fancy).

$ curl /api/chat  images:[wav]   → 200  "The gray fox jumped over the lazy dog."
$ curl /api/chat  audio:[wav]    → 200  prompt_eval_count: 15   # the sound never arrived
$ curl /api/embed audio:[wav]    → 501  "This server does not support
                                        embeddings. Start it with --embeddings"

Look at the middle line twice. The audio field doesn't error. It doesn't 400, doesn't warn. The server happily accepts your sound and then counts fifteen prompt tokens — the text prompt alone — and answers a completely different question. Our "transcribe this" request got a reply about explaining concepts, because as far as the model knew, no audio existed. A field that pretends to listen is worse than one that refuses.

How do you send audio to the ollama API?

Through the vision field. The multimodal runner behind ollama routes images bytes through the media encoders the model ships — and this model's encoder is an audio encoder (the load log literally prints has audio encoder). The field name lies; the pipeline doesn't.

import base64, json, urllib.request

wav = base64.b64encode(open("speech.wav", "rb").read()).decode()
req = urllib.request.Request(
    "http://localhost:11434/api/chat",
    data=json.dumps({
        "model": "lfm3-audio:2b",
        "messages": [{"role": "user",
                      "content": "Transcribe this audio exactly.",
                      "images": [wav]}],  # sound rides the picture door
        "stream": False}).encode(),
    headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["message"]["content"])

verify: returns the transcript string; swap images for audio and you get a 200 with prompt_eval_count: 15 — the audio ignored. Last tested 2026-11-02.

• ollama ≥ v0.40.13 — the LFM3 runner generation
• lfm3-audio:2b pulled (1.7 GB)
• a 16 kHz mono wav — the card's encoder wants 16 kHz; ours came from espeak-ng resampled down

The honest column — what one clip costs at the door. Audio tokens are the prompt total minus the ~7 text tokens of "Transcribe this audio exactly." — division, not vibes:

Roughly 14–15 tokens per second of speech, on two clips that only agree on being English. Budget a 10-minute meeting recording at about 9,000 prompt tokens before the model says a word.

How good is the transcription?

Good enough to be dangerous. Our second clip was espeak saying "Seven gigabytes of VRAM cost one hundred and nine dollars." The model returned: "Sam and Gigamite's unram cost 109 dollars." The number survived — 109 — and everything technical around it became a prog rock band. The fox clip came back as "The gray fox jumped over the lazy dog": right shape, wrong adjective, and a warm rerun repeated the same wrong adjective, so it's a stable opinion, not noise.

Fairness cuts both ways: synthetic TTS voices are the unfriendliest input a transcriber can get, and 16 kHz espeak is precisely nobody's real recording. Treat these as the floor, not the ceiling. The structural finding doesn't move either way — the model thinks before it transcribes. Every answer carried a thinking block. "Say hi in three words" burned 472 eval tokens to decide between "Hi, good morning" and "Hi there" — a committee meeting for a wave. Our October LFM3 run logged the same tax on the text side: 288 eval tokens for the two-word answer "File Size". The ear thinks like the mouth does.

What does audio cost in tokens?

The wall-clock bill, warm loads, hand-divided (the API's rate fields came back null on our October run too — we printed the division instead):

Sixteen-something tokens a second on two threads of a desktop CPU — right in the family's usual band. The receipt nobody prints is the ratio: for the two-word transcript, thinking tokens out-numbered the answer roughly three to one.

Why is there no audio field that works?

Because the plumbing arrived before the signage. LFM3-Audio landed in the library four days before we tested it; the API grew a media router for vision and reused it for the ear. The audio key exists in the wild enough that clients will reach for it first — and it does nothing, loudly quietly. The docs' only documented sound lane, the embeddings input, needs a server flag most installs never pass. So the working path for local speech to text on ollama today is a base64 wav in a field named for pictures — the ear is the picture door.

The honest ledger of what we couldn't test: no real microphone audio (synthetic voice only), no GPU box (these receipts are CPU-shaped and stay that way), and no second audio model to say whether the picture-door behavior is an LFM3 quirk or ollama's whole multimodal router. That last one is the receipt we want next.

Until the API grows a real ear field, send your sound through the picture door — and bill for the committee meeting.