Contrarian
llms.txt: what it actually does
Google says you do not need one. Anthropic recommends them. Both positions are correct, because they are answering different questions.
Google says you do not need one. Anthropic recommends them. Both positions are correct, because they are answering different questions.
llms.txt does not affect whether Google cites you in AI Overviews or AI Mode — Google has said so explicitly — but it is increasingly relevant to agents that read your site on a user’s behalf. It is an agent-readability file, not a ranking file. Vendors selling it as an AI visibility lever are selling the wrong thing; people dismissing it entirely are a year behind.
This is the question we get asked most often after an audit, usually because somebody has been told llms.txt is the fix. It is worth separating what the evidence supports from what is being sold.
Google’s position is unusually direct for Google. Its Search guidance on AI optimisation, updated 15 May 2026, states that llms.txt is not needed for AI Overviews, AI Mode, or any other generative AI Search feature, and groups it with tactics that do not help with Google AI visibility.
John Mueller has been blunter, comparing llms.txt to the keywords meta tag — a comparison with a specific sting, since the keywords tag is the canonical example of a signal search engines stopped honouring because it was trivially gamed. He also noted that no AI services were using it and that bots were not requesting the file. Gary Illyes and Amir Taboul confirmed at Search Central Live Deep Dive Asia Pacific that Google was not pursuing it.
If your goal is a citation in a Google AI answer, that is the end of the argument. No amount of llms.txt work moves that number.
The picture outside Google search is different, and it is moving:
The distinction that resolves the contradiction: search systems build their own index and have no reason to trust a file you wrote about yourself. Agents arrive at your site with a task, no index, and a narrow context window, and a clean map of what is where is immediately useful to them. Google is answering the search question. Anthropic is answering the agent question.
The honest cost-benefit. A good llms.txt takes an hour or two to write and costs nothing to maintain if your site is small. It will not get you cited today. It may matter for agent traffic, and that traffic is growing. Write one, place it correctly, and then go fix something that does move citation — starting with whether retrieval agents can reach you at all.
Most published examples are padded. The file is a map, not a brochure:
What it should not contain: sales copy, claims you cannot support, or anything that disagrees with the pages it points at.
| Work | Effect on citation | Effort |
|---|---|---|
| Unblock retrieval agents in robots.txt | Decisive where broken — a blocked page cannot be cited at all | Minutes |
| Add liftable answers under literal buyer questions | High | Days, ongoing |
| Organization and Person schema with stable identifiers | Moderate, compounding | Hours |
| Publish llms.txt | None for search today; relevant for agents | An hour or two |
We publish our own llms.txt and would not remove it. We also would not sell it to you as the reason you are invisible.
Every figure above is linked to the page that published it. Where a number is self-reported by the company that benefited from it, this post says so.
No. Google's Search guidance updated 15 May 2026 states llms.txt is not needed for AI Overviews, AI Mode or any other generative AI Search feature, and groups it with tactics that do not help AI visibility. John Mueller compared it to the keywords meta tag.
No, but it is an agent-readability file rather than a ranking file. Anthropic recommends it in its writing-for-agents guidance, OpenAI maintains llms.txt files for its Agents SDK and Agentic Commerce Protocol, and Chrome's Lighthouse 13.3 audits for it as part of agentic browsing readiness.
A quotable one-paragraph description of the organisation, explicit disambiguation from things with similar names, a short annotated list of key pages, definitions of core terms, and identity facts that match the site's structured data exactly. It should not contain sales copy or unsupported claims.
No. Check whether retrieval agents such as OAI-SearchBot, ChatGPT-User and PerplexityBot are allowed to fetch your pages. A blocked page cannot be cited at all, and that check takes minutes.
