Nine failures that recur on real sites, in the order they cost you the most.
The most common AEO mistakes are blocking retrieval agents by accident, burying the answer below preamble, rendering content only in JavaScript, describing the company differently in every place, and measuring with a single run.
The same handful of failures turn up again and again, and almost none of them are about writing quality. They are configuration errors, structural habits carried over from SEO, and one measurement mistake that makes all the others invisible. The encouraging part is that most of them are cheap to fix once somebody has actually looked.
One blanket rule that blocks retrieval agents alongside training crawlers. It is usually written in a single sitting, during a conversation about “AI scraping our content”, by someone who was right to be concerned about training and did not know the two classes of agent exist.
They do, and the consequences are completely different. A training crawler collects content that may be used in building a model; the effect of blocking it is diffuse and arguably in your favour. A retrieval agent fetches your page in the seconds after a user asks a question, so an assistant can read it and quote it. Refuse that one and you are not protecting anything — you have simply removed yourself from the answer. The two classes, named agent by named agent →
The version that causes the damage looks like this:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: PerplexityBot
Disallow: /The first rule is a decision. The next three are self-harm, and whoever wrote them almost certainly did not intend the difference.
Check who wrote your robots.txt and when. On most sites the AI rules were added once, quickly, during a security or legal conversation, and nobody has read them since. If your marketing team is running an AI visibility project while a rule from eighteen months ago refuses the search agents, nothing downstream of that rule can work. Read the file before you commission any other work.
Because the block is at the edge, not in the file, and this is the mistake that survives the longest precisely because it is invisible to everyone involved.
Content delivery networks added AI-bot categories to their bot-management products. Some are switched on by default, some are enabled during a routine security review by an engineer who was thinking about scrapers rather than search. The rule lives in a dashboard, not in your repository, so no code review will surface it and no developer reading the codebase will find it. Your robots.txt says allow. Your edge says deny. The agent gets the edge’s answer.
The only way to settle it is to make the request as the agent and read the status code:
curl -s -o /dev/null -w "%{http_code}\n" \
-A "OAI-SearchBot/1.0; +https://openai.com/searchbot" \
https://yourdomain.com/your-key-pageA 200 is a pass. A 403 is a refusal and a 429 is a throttle, which behaves like a refusal from the agent’s side. Test a deep page as well as the homepage, because rules are frequently scoped to paths, and check the CDN bot settings directly rather than inferring them from the file.
It is the most common content mistake, and it comes straight from SEO habits that used to be rewarded. The page does contain the answer. It appears four paragraphs after the heading, behind scene-setting, a restatement of the question and a note about why the topic matters.
Retrieval works on passages. What gets lifted is a short span of text that reads as a complete claim, and if the span directly under your heading is throat-clearing, that is what represents the section. Consider the difference:
| Buried | Extractable |
|---|---|
| “Onboarding is something we take seriously, and every business is different. Over the years we have refined an approach that…” | “Onboarding takes two weeks and covers access, schema and your five highest-intent pages.” |
| “Pricing is a question we get asked a lot, and there is no simple answer, because it depends on several factors…” | “Plans start at a fixed monthly fee per seat and include SSO on every tier.” |
The fix is mechanical. Phrase the heading as the question a buyer would type, answer it in the first forty words underneath, and put the context afterwards. The build-up can follow the answer. It cannot precede it.
Frequently, and it is the failure most likely to be dismissed as impossible by the people who built the site — because in a browser everything looks fine.
The test is not what a browser shows. It is what comes back in the raw response. Fetch the page as an agent and search the response for a sentence you expect to find:
curl -s -A "OAI-SearchBot/1.0" https://yourdomain.com/key-page | grep -c "a sentence from the page"A count of zero means your content was not in what the agent received, however well it renders for a human. A 200 containing an empty application shell is a pass on access and a failure on everything after it.
Not every agent executes JavaScript, and those that do may not wait for slow client-side data. This bites hardest on exactly the pages that matter: pricing tables assembled at runtime, comparison grids driven by a content API, FAQ accordions whose answers load on expand. Server-render or statically generate anything you want quoted.
Because an assistant will not confidently assert something about an entity it cannot pin down, and inconsistency is how a company becomes unpinnable.
The pattern is nearly universal on sites that have been alive for a few years. The schema description was written at launch. The footer line was rewritten during a redesign. The llms.txt was added last year by someone quoting the pitch deck. The LinkedIn bio is different again. Each one is defensible; together they give a model four candidate descriptions of you, agreeing on nothing specific.
Models quote these strings close to verbatim, which is why the fix is unusually high-leverage for the effort involved: write one sentence that says what the company does and who for, and make it the identical sentence in your Organization schema, your llms.txt, your footer, your about page and every third-party profile you control. Give the Organization a stable identifier that does not regenerate per template, add a real Person node for the founder with links that resolve, and disambiguate explicitly if your name collides with another company or a generic term.
A quick diagnostic: ask an assistant what your company does. If the answer is wrong, read your own structured data before blaming the model. It is usually repeating one of your four descriptions back to you. What has to be true before you are named →
Yes, and this one surprises people, because the same sentence was rewarded in SEO for years.
“Studies show that most buyers research online before contacting a vendor” is a sentence an assistant cannot safely repeat. Repeating it means asserting a claim with no source attached, and the systems that produce citations are built to avoid exactly that. The figure is not quotable, and the paragraph around it is weakened by association.
The same claim with a named source, a publication and a date is quotable, because an assistant repeating it can attribute it. So either attribute the number properly or make the point without it — “most buyers research before they contact anyone” as an observation is honest, unremarkable and costs you nothing.
The rule extends further than statistics. Invented testimonials, unverifiable results and metrics presented as measured when they were estimated all fail the same way, and they carry a second risk: a buyer who checks and finds nothing behind the number has learned something about you that no amount of visibility will fix.
Publishing one is not. Treating it as the project is, and that substitution happens constantly, because llms.txt is a file you can ship in an afternoon and tick off, while crawler access requires talking to whoever owns the CDN.
Its status is genuinely contested. Google has said publicly that it is not used for its AI features, while other vendors recommend it and some agents do request it. The proportionate position: it costs almost nothing, ship it, and do not let it displace the work with demonstrable effects.
A site that publishes a beautifully maintained llms.txt while returning 403 to OAI-SearchBot has done the equivalent of putting a sign on a locked door. The order matters more than the file does. The checks in the order that matters →
Because assistant output is non-deterministic. Ask the same question twice and the cited sources can differ with nothing on your site having changed. A single check is a screenshot, not a measurement, and a company measuring this way is unable to tell a real improvement from ordinary variation.
That has a knock-on effect worth naming. Teams that measure with single runs end up believing whichever result they saw most recently. A good run on Tuesday becomes “we are visible in ChatGPT”, the project is declared finished, and the crawler block that has been quietly refusing the search agent the whole time is never found — because the one favourable run came from a logged-in account with memory of the company.
The minimum honest protocol: a frozen set of buyer questions, at least three runs per prompt, logged out, with the market pinned to where you sell, recording how often you were cited rather than whether you ever were. Report the spread and note the model version. The full protocol →, and the one we publish and run →.
Rarely, and this is the most expensive mistake on the list because it consumes budget for months before anyone concludes it is not working.
The reasoning is inherited from a different game: more pages, more coverage, more chances to rank. Retrieval does not work that way. What gets cited is a passage that answers a specific question in a form that survives being lifted out of the page, and forty articles about industry trends contain no such passage. Worse, thin pages published at pace can make an entity harder to resolve rather than easier, by scattering inconsistent descriptions of the company across a larger surface.
The better use of the same effort is narrow: rewrite the pages where buying questions are actually answered — pricing, comparisons, integrations, use cases — so that each has a heading phrased as the question and an answer in the first forty words. Then spend what is left on corroboration elsewhere, because being named in sources an assistant already trusts does more than another post on your own domain.
Which leaves the question of order. These come first, and between them they are about a week of work:
robots.txt.llms.txt, footer and external profiles, with a stable identifier.The first four are typically a week of work between them and account for the majority of cases where a company cannot understand why it is absent. If you would rather find out which of the nine apply to your site than work through the list yourself, the free AI visibility check tests access per named agent, reads your pages the way a retrieval agent does, and reports who is cited in your place. No citation is ever promised — model output is not deterministic and nobody controls it — but the reasons you are currently uncitable can be found and removed.
Blocking retrieval agents, usually by accident, through one blanket rule written during a conversation about AI scraping. Nothing else you do matters while the agent is refused, and because the block is often at the CDN rather than in robots.txt, most teams never find it in their own codebase.
Request your own page while identifying as each named agent and read the status code; a 403 or 429 is a refusal. Do it for a deep page as well as the homepage, since rules are often scoped to paths, and check your CDN's bot-management settings directly.
It costs almost nothing and some agents request it, so publish it. The mistake is treating it as the project. Google has said publicly that it does not use it for AI features, and a perfect llms.txt on a site that refuses retrieval agents changes nothing.
Usually one of three reasons: their pages are reachable and yours are not, their sentences survive being quoted and yours need surrounding context, or their company resolves as a clearer entity. Content quality is the third thing that matters, not the first.
Only the unattributed ones. A figure with a named source, publication and date is quotable, because an assistant repeating it can attribute it. A figure introduced with “studies show” is not, and the surrounding paragraph is weakened with it.
No, and any vendor promising that is misdescribing how the systems work. Model output is non-deterministic and nobody controls which sources are selected. What fixing these does is remove the reasons you are currently uncitable, which is the only part anyone can actually control.