bingbot user agent — the biggest bot your census misses
Bingbot filed 1,028 pageviews here in seven days — 5.1× Googlebot's 203 — and every named census missed it, because it walks in wearing Mozilla.
Bingbot's user agent is
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36. The name you grep for sits in the middle of the string, not the front — so a census that groups by the first token never sees it. On this site, Bingbot out-crawled every named bot in the week measured: 1,028 visits against Googlebot's 203. Countbingbotby substring, and the biggest bot walks out of the anonymous pile.
The count below is live, not recalled: one week of this site's traffic lake, measured the morning this ran. Every number in it names its basis — the tracker classifies crawlers by user agent, and the class layer got Bingbot right (seo_bot). The named layer is what blew.
What is the bingbot user agent string?
The whole claim sentence is short: Bingbot's real user agent is a Chrome-shaped string with compatible; bingbot/2.0 in the middle, and the keyword is the only place its name appears. Microsoft's own crawler documentation lists the shape; here is the exact string this site caught, verbatim from the traffic lake:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36verify: the string above is copied byte-for-byte from this week's measured spans, not from Microsoft's docs — docs describe the shape, the lake keeps the specimen.
Two things hide inside it. First, it starts with Mozilla/5.0 — the same opening as every browser and half the scraper fleet on the web. Second, the version that matters is buried after a semicolon, where a naive parser splitting on spaces and slashes reads the first token as the bot's name. A census that does that ends up with a bucket called Mozilla/5.0 holding Bingbot's 1,028 visits next to twelve thousand genuinely anonymous hits.
Why your bot census misses bingbot
The mechanism is one line: name-keyed counting works for bots that name themselves at the front, and Bingbot deliberately does not. PetalBot, GPTBot, ClaudeBot and PerplexityBot all announce themselves up front — which is why the census post this site ran on Huawei's PetalBot could name it cleanly: four robots.txt asks in 28 days, and it read everything else on faith. Bingbot's UA follows the classic browser-compatible pattern instead, the same pattern that let it ride through decades of user-agent sniffing unbothered.
| Bot | First-token census | Keyword census | Where the difference comes from |
|---|---|---|---|
| Bingbot | 0 (files as Mozilla/5.0) | 1,028 | UA opens Mozilla/5.0, keyword mid-string |
| meta-externalagent | 949 | 949 | self-declared at the front |
| PetalBot | 748 | 748 | self-declared |
| Applebot | 277 | 277 | self-declared |
| Googlebot | 203 | 203 | self-declared |
| PerplexityBot | 185 | 185 | self-declared |
| OAI-SearchBot | 162 | 162 | self-declared |
| ClaudeBot | 149 | 149 | self-declared |
| GPTBot | 144 | 144 | self-declared |
| Bytespider | 75 | 75 | self-declared |
Same week, same lake, two counting methods. The class layer (search-bot vs AI-bot vs other) got Bingbot right all along — its 1,028 visits sit inside the seo_bot class of 4,211, about one visit in four. Only the name column lied.
That is the uncomfortable part: the class layer was right and the name layer was blind, and the mistake is invisible in the output. A dashboard that says Mozilla/5.0: 12,000 looks like noise, not like the single largest measured crawler on the property. The party was going fine; nobody noticed the guest of honour came in a paper bag.
How to count bingbot on your own site
The fix costs one line anywhere you keep logs: match on the substring, never on the leading token.
verify: last tested 2026-11-17 — the census numbers above were re-measured in this site's traffic lake the same morning this ran; the grep shape is standard log hygiene and the reverse-DNS step cites Microsoft's documented verification method.
And the honest ledger, wound open: our own census posts had this same blind spot. The PetalBot piece named its league table correctly and still would have told you Googlebot was the biggest search bot here — off by five to one. The earlier crawlers-versus-search-crawlers census had it too. The fix rides forward in every count from now on; the old numbers stay printed exactly as they were measured, because a correction that quietly rewrites history is just a second kind of costume. One thing held, though: the week Bingbot came dressed as Mozilla, it still minded its manners — and the IndexNow pings this site sends were answered, on time, every time.