llms.txt checker — grade your file before the bots do

the grammar is one page of markdown

The spec asks for almost nothing. Exactly one # title on the first line. An optional > blockquote right under it — the two-sentence pitch an agent quotes when it cites you. Then ## sections, each holding links shaped - [title](url): description, the description telling a machine why the link earns its place. That is the whole contract, and it exists because answer engines read differently than people: a crawler decides in seconds what your site is and which three pages matter, and the file is your one chance to hand it that answer instead of a sitemap and a prayer.

Our grader prices the contract the way a strict editor would. Start at 100. A structural fail — no H1, no sections, a section carrying zero links, bidi control characters hiding in the bytes — costs 25. A polish miss — missing blockquote, links floating outside sections, http:// URLs, duplicates — costs 8. Ninety or better ships; seventy to eighty-nine is close; forty to sixty-nine is rough; below forty, rebuild from the skeleton.

ten real files, graded

We pulled /llms.txt from ten sites on November 13 with a plain HTTPS fetch — no special access, the same request your browser makes. The holes are the story: Anthropic, Hugging Face and simonwillison.net return 404 — no file at all. OpenAI's edge answers 403 — the path exists and is gated shut. DuckDuckGo serves an HTML error page at the path, which is worse than nothing: a parser that trusts the URL gets markup where markdown should be. Five sites serve a real file, and they average 87.

site	file served	grade	verdict	first miss
mistral.ai	yes	100	SHIP IT	—
answer.ai	yes	100	SHIP IT	—
stripe.com	yes	92	SHIP IT	one duplicate URL
cloudflare.com	yes	75	CLOSE	one zero-link section
mrsaynothing.dev	yes	67	ROUGH	six zero-link sections, no summary
anthropic.com	no — 404	—	—	no file
huggingface.co	no — 404	—	—	no file
simonwillison.net	no — 404	—	—	no file
openai.com	gated — 403	—	—	path unreadable
duckduckgo.com	html, not text	—	—	wrong bytes at the path

census run 2026-11-13, one plain GET per site, graded by the exact parse/check/score logic shipped in the tool.

how the grade is priced

Inside, the grader is as boring as a tax form, which is the point. It parses the file into the four things the spec defines — title, summary, sections, links — then runs nine checks over the parse and subtracts from 100. One check deserves a confession: the bidi and zero-width character test exists because some submission pipelines force-pend any file carrying them. We learned that shipping our own Arabic twins — three invisible control characters, one stuck post. Your file can carry that landmine too; the grader finds it before a machine does.

our own file flunked first

Here is the part we could have edited out and did not: we built a smoke detector, and our own kitchen was the first fire. This site's llms.txt is an endpoint map, not a link catalog — seven sections, one link, no blockquote summary. Six sections carry zero links, the summary is missing, and the grade lands at 67, ROUGH. The failures are honest ones: when we asked what an llms.txt even is, our piece was about the door being unread — this file is written for agents, just not in the spec's grammar. The fix list is already queued for the next site pass, which is the correct order of operations: grade first, edit second, publish the receipts regardless.

grade: 67 — ROUGH
  ✗ sections with zero links: Identity, Languages, Projects,
    Corpus, Machine lanes (agents: start here), Agent doors
  ! no blockquote summary — agents read it as the site's pitch
  ✓ exactly one H1, on line 1
  ✓ 7 ## sections · 1 link · 0 duplicates · 0 bidi chars

ship yours tonight

The grader lives at github.com/mrsaynothing/llms-txt-checker, served as one static page — no build, no upload endpoint, no analytics; your file never leaves the tab. Paste it, or fetch a live one straight from the box, and you get the grade, the per-check receipts, and a fix list ordered by cost: the 25-point failures first, the 8-point polish after. When it is beyond saving, the skeleton button hands you a clean file to rebuild from, which takes about four minutes. The ten-site census above took ten minutes, and three of those sites are still missing the file entirely — the cheapest SEO left on the web is a page of markdown nobody has written yet.

• 1 one # title, on line 1, matching your site name
• 2 a > blockquote under it: what the site is, what an agent should do first
• 3 every link inside a ## section — no strays
• 4 every link carries a : description — the why, not the what
• 5 zero bidi or zero-width characters; https only; no duplicates

The answer engines are already crawling — our own logs caught PerplexityBot reading this site's robot signs for weeks before anyone wrote it a proper door. The file is small, the grammar is one page, and the grade is free. Grade your llms.txt before the bots do it for you.