The evidence that llms.txt does not earn citations

llms.txt does not earn AI citations. Two 2026 studies covering close to half a million domains between them found no measurable relationship between publishing the file and getting cited by an AI engine, and most of the files that exist are never read by a crawler at all.

The idea was reasonable enough to try: a plain markdown file at the root of a domain, listing a site's best pages for a language model to read directly instead of parsing HTML. Since the format was proposed in late 2024, hundreds of generative engine optimisation guides have told founders to publish one as step one of an AI search strategy. Our own August 2026 audit of what actually correlates with getting cited named six factors: crawlability, correct HTTP delivery, page structure, schema markup, freshness and original data. llms.txt was not on that list. By September 2026 there was enough independent, domain-scale data to say plainly why.

What the adoption data shows

SE Ranking's November 2025 study of roughly 300,000 domains found that 10.13% had published an llms.txt file, with no measurable citation benefit from doing so.

Adoption held almost identical across traffic tiers: 9.88% among low-traffic domains, 10.54% among mid-traffic domains, 8.27% among the highest-traffic sites in the sample. That rules out the tidy explanation that only small, under-resourced sites skip it. When SE Ranking's analysis removed the llms.txt variable from its citation prediction model, the model's accuracy improved. The file was not neutral in that model. It was adding noise.

What happens to the file once it exists

Ahrefs analysed 137,210 domains in June 2026 and found that 97% of published llms.txt files had received zero recorded AI bot requests.

About 28% of the domains in Ahrefs' study published a valid file, a rate its own authors flagged as skewed upward because their analytics customers are more technical than the open web. Of the roughly 38,000 valid files it found, only around 1,100 had ever received a single request from anything, bot or otherwise. In Ahrefs' sample, AI retrieval crawlers, the ones actually gathering material to cite, accounted for about 1% of the requests those files did get. The largest single requester was SEO audit tooling checking whether the file existed, not an AI engine reading it.

A separate server-log analysis by Limy.ai measured more than 515 million AI bot requests across a 90-day window and found that 408 of them touched llms.txt, across GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended combined.

What OpenAI and Anthropic's own documentation actually says

Neither company lists llms.txt as a mechanism for directing its crawlers. Both describe robots.txt as the tool that does that job.

OpenAI documents three separate crawlers, GPTBot for training, OAI-SearchBot for ChatGPT search, and ChatGPT-User for live user requests, and its own published guidance tells site owners to manage them through robots.txt entries. Anthropic runs the equivalent three, ClaudeBot, Claude-SearchBot and Claude-User, and its help centre gives the same instruction: add a User-agent block and a Disallow line to robots.txt to opt out. llms.txt appears in neither company's crawler documentation. That does not prove the file is actively ignored. It does mean the two labs building the crawlers publish no reason to believe it is read, which is a strange gap for a spec pitched as being written for language models to consume.

Where llms.txt actually does something

The file has a real job. It is just not the one most GEO advice sells it for.

Stripe, Vercel and Cloudflare all publish an llms.txt file alongside their API documentation, and coding assistants such as Claude Code and Cursor do read files like theirs to load documentation context without crawling a full docs site. That is a genuine, working use case: feeding a coding agent an API reference while it writes code against it.

That is a different job to earning a citation in an AI search answer, and the two get conflated constantly. A file built to help a coding agent find an API reference is not doing the same job as one built to get a marketing page quoted in a ChatGPT answer. Every study above measured the second job. On the first, llms.txt earns its place, which is exactly why the confusion is so persistent: the format works well enough in one context that its advocates keep recommending it in the other.

Why the advice to publish one persists anyway

Publishing an llms.txt file feels like doing something about AI search at almost no cost, which is exactly the profile of advice that spreads regardless of whether it works.

It takes ten minutes to write, it cannot break anything on the page it sits next to, and it gives a founder something concrete to point at when a board asks what the team is doing about AI visibility. Recommending it also costs the person making the recommendation nothing, so it keeps appearing in checklists years after the evidence turned against it. Cheap to do, cheap to recommend, hard to disprove without running a domain-scale study of your own: that combination is exactly the shape of advice this industry produces too much of.

What we do with ours

We keep our own llms.txt file because it costs nothing to maintain, and we tell clients plainly that it earns them nothing on its own.

It is technically correct and it is not going to hurt anything, so there is no reason to take it down. What we do not do is spend a client hour on it, put it near the top of a GEO audit checklist, or let a founder believe that publishing one has moved their citation rate. That hour is better spent on the things our own audit actually found moving the number: fixing how pages get delivered to a crawler in the first place, building schema an engine can parse, and working through the seven things that actually decide whether an AI engine cites you. Our AI Search Visibility Field Manual walks the full list in order of what to do first.

llms.txt is not a lever. Keep the file if you already have one, skip the hour if you do not, and spend it on the six things that are actually proven to move an AI citation rate.