What is llms.txt, and do you need one?
Last updated:
Short answer: llms.txt is a proposed file that tells AI systems what your site is about, in plain Markdown, at /llms.txt. It is a community proposal, not an official standard. Google says Google Search ignores it, and no other major AI company has said it reads it. It takes minutes to add and does no harm, so add one only if it is quick for you. It will not change your rankings.
What it actually is
llms.txt was proposed by Jeremy Howard and published on 3 September 2024. The specification is published at llmstxt.org. Version 2 was published on 10 August 2026. It now describes itself as a proposal “to standardise on using an /llms.txt file to provide information to help agents use a website.” An agent here means an AI assistant acting for a person.
The problem it addresses is real. When an AI assistant fetches one of your pages, it gets HTML full of navigation, cookie banners, scripts and boilerplate. It can only read so much at once. An llms.txt file offers a clean, short summary instead: here is what this site is, here are the pages that matter.
Where it goes and what it looks like
The file lives at the root path /llms.txt (the specification also permits a subpath). Unusually for a machine-readable file, it uses Markdown rather than XML or JSON. The specification notes it “uses Markdown to structure the information rather than a classic structured format such as XML.”
The specification defines this order:
- An H1 with the project or site name. This is the only required section.
- A blockquote containing a short summary.
- Optional Markdown sections giving more detail.
- Optional H2-delimited sections containing lists of links, each optionally annotated with a note.
A minimal example:
# Acme Roofing > Roof repair and replacement for homes in Leeds, established 1998. Family-run, fully insured, free quotes within 24 hours. ## Main pages - [Services](/services): repairs, replacement, guttering and emergency call-outs. - [Pricing](/pricing): typical costs by job type. - [Contact](/contact): phone, email and the areas we cover.
Does it actually work? The honest answer
This is where most articles on the subject overstate things, so here is what can and cannot be verified:
- It is a proposal, not a standard. The specification says so plainly, and states that it “is open for community input.” No standards body has adopted it.
- Google says Google Search ignores it. Google's guide to generative AI features is direct about it. It says these files “will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.” Google's guide.
- No other major AI company has said it reads llms.txt. As of this writing, the crawler pages of OpenAI, Anthropic and Perplexity do not say their crawlers or assistants read it. OpenAI and Perplexity do publish llms.txt files for their own documentation, which is a different thing. Anyone telling you ChatGPT “uses your llms.txt” is claiming something these companies have not said.
- Adoption so far is mostly tools that write the file. The proposal lists platforms and plugins that generate it, such as Yoast SEO, Wix and GitBook. That is real adoption, but it is not the same as the large AI assistants reading it.
So should you add one?
Only if it is quick for you, and with clear eyes about why. It is a small text file that costs nothing to serve. It does not change your rankings. If more AI tools start reading it, you are already there. Treat it as a small bet, not an SEO tactic.
What it will not do: improve your Google rankings, guarantee a mention in ChatGPT, or substitute for having your content actually indexed and crawlable. If your pages are not reachable and readable in the first place, llms.txt changes nothing.
One small caveat is worth knowing. The file llms.txt is normal and publicly reachable. So if anything links to it, a search engine can index it like any other page. If you would rather it did not appear in search results, serve it with an X-Robots-Tag: noindex header.
How to write a good one
- Lead with what you do and who for, in one sentence. Assume the reader has no context.
- Link only pages that matter. A curated list of five beats an exhaustive list of fifty. An AI tool can only read so much at once, so the short list is the point.
- Annotate each link with a short note on what the reader will find there.
- Include concrete facts: location, founding year, pricing model, contact address. Specifics are what get quoted back.
- Keep it truthful and current. A file that contradicts your actual pages is worse than none.
llms.txt is not robots.txt
They are unrelated and solve different problems. The robots.txt file is a long-standing convention. It controls whether a crawler may fetch your pages. The llms.txt proposal describes what your site is, for a system that is already reading it. Do you want to control which AI crawlers can access your site at all? That is a robots.txt question. See our guide to AI crawlers and robots.txt.
Checking yours
Our free checkup looks for /llms.txt as part of its AI Search Readiness category. It also checks the things that matter more. Does your content render without JavaScript? Are your headings structured? Do you have valid structured data? You can see the full list of checks, or just run one.