Using it
Paste your sitemap.xml, or one web address per line. Pages are grouped into sections by the first part of their address, so /guides/… pages go under Guides, and pages at the top level go under Pages. Each link is named from its address.
To set the details yourself, start with a line like # Your site name and a line starting > with a one-sentence summary. Any address can be followed by | a title | a note, as in the sample. To check an llms.txt you already have, set From to llms.txt and To to Check report.
Nothing is fetched. The page can't crawl your site without sending your address somewhere, so it works only from what you paste.
What llms.txt is
llms.txt is a proposal from Jeremy Howard, made in September 2024 and published at llmstxt.org, for a markdown file at the root of a website that gives language models a short, clean map of it. A model or an AI tool can read one small file instead of trying to make sense of every page's menus, adverts and scripts.
The format is plain markdown in a fixed order: an H1 with the site's name, which is the only part required; a blockquote with a short summary; any other notes, with no headings; then H2 sections, each a list of links written as - [name](url), with an optional colon and a note. A section called Optional is for links a tool can skip when it needs to keep things short.
The parts, in order
| Part | Written as | Required? |
|---|---|---|
| Site name | # Name | Yes |
| Summary | > One or two sentences | No |
| Notes | Paragraphs or lists, no headings | No |
| Sections of links | ## Section, then - [name](url): note | No |
| Skippable links | ## Optional | No |
Making it useful
A generated file is a start. Cut it down to the pages that answer real questions, rename links to what people call them, and add notes that say what each page is for. The proposal also suggests offering a plain markdown copy of important pages at the same address with .md added, so tools can read them without the HTML.
Questions
Where does llms.txt go?
At the root of your site, such as https://example.com/llms.txt, though the proposal allows it at any path.
Do AI companies read llms.txt?
It's a proposal, not a standard any of them has committed to, so treat it as a cheap extra rather than a ranking factor.
Is llms.txt the same as robots.txt?
No. robots.txt says what crawlers may fetch; llms.txt describes what's worth reading. One doesn't replace the other.
Sources
Added . What's new