What is llms.txt, and do you need one?
Last updated:
Short answer: llms.txt is a proposed file that tells AI systems what your site is about, in plain Markdown, at /llms.txt. It is a community proposal rather than an official standard, and no major AI company has publicly confirmed that it reads the file. It takes minutes to add and carries essentially no downside - so it is a reasonable low-effort bet, not a ranking lever.
What it actually is
llms.txt was proposed by Jeremy Howard and published on 3 September 2024. The proposal describes itself as a way “to standardise on using an /llms.txtfile to provide information to help LLMs use a website at inference time.” The specification is published at llmstxt.org.
The problem it addresses is real. When an AI assistant fetches one of your pages, it gets HTML full of navigation, cookie banners, scripts and boilerplate, and it has a limited context window to work with. llms.txt offers a clean, curated summary instead: here is what this site is, here are the pages that matter.
Where it goes and what it looks like
The file lives at the root path /llms.txt (the specification also permits a subpath). Unusually for a machine-readable file it uses Markdownrather than XML or JSON - the specification notes it “uses Markdown to structure the information rather than a classic structured format such as XML.”
The specification defines this order:
- An H1 with the project or site name. This is the only required section.
- A blockquote containing a short summary.
- Optional Markdown sections giving more detail.
- Optional H2-delimited sections containing lists of links, each optionally annotated with a note.
A minimal example:
# Acme Roofing > Roof repair and replacement for homes in Leeds, established 1998. Family-run, fully insured, free quotes within 24 hours. ## Main pages - [Services](/services): repairs, replacement, guttering and emergency call-outs. - [Pricing](/pricing): typical costs by job type. - [Contact](/contact): phone, email and the areas we cover.
Does it actually work? The honest answer
This is where most articles on the subject overstate things, so here is what can and cannot be verified:
- It is a proposal, not a standard.The specification says so plainly, and states that it “is open for community input.” No standards body has adopted it.
- No major AI company has publicly confirmed it reads llms.txt.As of this writing, neither OpenAI, Anthropic, Google nor Perplexity documents llms.txt in their crawler or publisher guidance. Anyone telling you ChatGPT “uses your llms.txt” is asserting something none of these companies have said.
- Adoption so far is mostly developer-tooling. The proposal cites projects and plugins that generate the file; that is genuine adoption, but it is not the same as the large assistants consuming it.
So should you add one?
Yes, if you can spare the time - with clear eyes about why. It is a static text file with no performance cost, it is not a ranking factor either way, and if adoption does arrive you are already there. Treat it as cheap optionality rather than as an SEO tactic.
What it will not do: improve your Google rankings, guarantee a mention in ChatGPT, or substitute for having your content actually indexed and crawlable. If your pages are not reachable and readable in the first place, llms.txt changes nothing.
One small caveat worth knowing: llms.txt is a normal, publicly reachable file, so if anything links to it, a search engine can index it like any other page. If you would rather it did not appear in search results, serve it with a X-Robots-Tag: noindex header.
How to write a good one
- Lead with what you do and who for, in one sentence. Assume the reader has no context.
- Link only pages that matter. A curated list of five beats an exhaustive list of fifty - the whole point is fitting into a limited context window.
- Annotate each link with a short note on what the reader will find there.
- Include concrete facts - location, founding year, pricing model, contact address. Specifics are what get quoted back.
- Keep it truthful and current. A file that contradicts your actual pages is worse than none.
llms.txt is not robots.txt
They are unrelated and solve different problems. robots.txt is a long-standing convention that controls whether a crawler may fetch your pages; llms.txt is a proposal that describes what your site is for a system that is already reading it. If you want to control which AI crawlers can access your site at all, that is a robots.txt question - see our guide to AI crawlers and robots.txt.
Checking yours
Our free checkup looks for /llms.txt as part of its AI Search Readiness category, alongside the things that matter more: whether your content renders without JavaScript, whether your headings are structured, and whether you have valid structured data. You can see the full list of checks, or just run one.