You do not need an llms.txt file to appear in Google's AI features, and no major engine has published a commitment to use one. llms.txt is a proposal, not a standard that Google or OpenAI document as a ranking or retrieval input. Adding one is cheap and harmless, but your time is better spent on crawler access, clear pages and measuring what the engines actually say.
This guide explains what llms.txt is, what the engines have said, and what to do instead.
What is an llms.txt file?
llms.txt is a proposed Markdown file placed at the root of a website (/llms.txt) that gives AI systems a short, curated overview of the site's content. It was proposed by Jeremy Howard and first published in September 2024. The idea is to spare an AI agent from parsing full HTML pages, which the proposal describes as costly and imprecise (llmstxt.org).
Under the proposal, the file contains:
- An H1 with the project or site name, which is the only required element
- A blockquote with a short summary
- Optional sections of detail
- H2 sections holding lists of links, each with an optional note
- An "Optional" section for secondary links that can be skipped
Here is an example for an example brand, not a real site:
# Rotabel
> Payroll software for restaurants, cafés and hotels.
## Product
- [Pricing](https://rotabel.example/pricing): plans and what each includes
- [Tips and tronc](https://rotabel.example/tips): how tipped pay is handled
## Compare
- [Rotabel vs Pennworth](https://rotabel.example/vs/pennworth)
Do Google and OpenAI use llms.txt?
Neither company documents llms.txt as something its search or AI features rely on.
- Google. Google's documentation on AI features says you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that no special schema.org structured data is needed. The same page says there are no additional requirements beyond being indexed and eligible to appear in Search with a snippet (Google Search Central).
- OpenAI. OpenAI's crawler documentation describes controlling its bots through robots.txt, with separate settings for
OAI-SearchBot,GPTBotandChatGPT-User(OpenAI).
Positions can change. Check these pages for the current wording before you decide, and treat any claim that "llms.txt boosts AI visibility" as unproven until you can test it yourself.
Should you add an llms.txt file anyway?
It is reasonable if:
- You maintain documentation or developer content and want a tidy map for AI coding tools and agents.
- You like keeping a concise, up-to-date summary of your key pages.
- You can maintain it, because an out-of-date map is worse than none.
Do not expect it to change what ChatGPT, Gemini or Google say about you. If you add one, treat it as an experiment. Note the date, then watch your results for several weeks. How to track your brand in AI answers shows how to measure it.
What should you do instead?
These steps have clearer support and bigger payoffs.
- Check crawler access. Make sure you are not blocking the crawlers that power AI search by mistake. See Can AI crawlers read your site?
- Make pages easy to quote. Put plain statements about who you serve, what you offer, what it costs (or where to see current plans) and how you compare near the top of the page. Google's own guidance for AI search is to offer unique, valuable content for people (Google Search Central blog).
- Keep facts consistent everywhere. Old prices or retired features on other sites can end up in answers. See AI citation gaps.
- Keep your SEO base healthy. AI features build on pages Google can crawl and index. See AEO vs SEO.
- Measure. Track the prompts that matter, daily, and see whether any change moves them. Start with What is AEO? if you are new to this.
Common questions about llms.txt
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which paths they may fetch, and it is defined by RFC 9309 (RFC 9309). llms.txt is a proposed content guide for AI systems. One controls access, the other would only suggest where to look.
Can llms.txt block AI training?
No. It is a list of pages you want read, not a permission mechanism. Training controls such as GPTBot and Google-Extended work through robots.txt.
Where does the file go?
At /llms.txt on your site root, or at a subpath such as /docs/llms.txt to cover the pages under it, according to the proposal (llmstxt.org).
Measure what the engines actually say
Whether or not you add a file, the real test is whether ChatGPT, Gemini, Google AI Overviews and Google AI Mode name your brand. Verte checks all four every day and shows the result in the morning with AI visibility tracking, and its site audit covers what AI crawlers can reach.
Start tracking your business or start as an agency. See current plans.