llms.txt: An Optional Discovery Layer for AI Crawlers — What It Is and Is Not
llms.txt is an emerging, optional text file that organizes a site's public resources for machine-readable discovery by AI crawlers. It is a supplementary convention — it helps AI systems find your documentation, features, and guides, but it does not guarantee AI rankings or citations.
llms.txt is a lightweight, human- and machine-readable index of a site's important public resources. It sits alongside robots.txt and sitemap.xml as a discovery aid oriented toward AI and LLM crawlers.
What to include
Describe the product briefly at the top, then list important public pages — homepage, features, documentation, guides, and API docs if any — grouped logically. Keep links current and accurate; stale URLs reduce the file's value.
What it is not
llms.txt does not guarantee that an AI system will rank, surface, or cite your content. It is a discovery convenience. Citing is still controlled by the AI provider's model and policies, not by the presence of a file.
| File | Purpose |
|---|---|
| robots.txt | crawl directives |
| sitemap.xml | indexable URL list |
| llms.txt | AI-oriented resource map |
Frequently asked questions
Does llms.txt guarantee AI citations?
No. It is a supplementary discovery convention. It may help AI crawlers find your resources, but citation and ranking are controlled by the AI providers, not by any on-page file.
Is llms.txt a standard?
It is an emerging convention, not an official standard. Treat it as a best-effort supplement to a clean sitemap and robots.txt.
About the author
RankoraXPress Team — Search Visibility Editorial
The RankoraXPress editorial team publishes original daily briefs on search visibility — covering SEO, AEO, GEO, and developer SEO. Every brief is written in-house and never scraped or copied from third-party sources.
Expertise: SEO, AEO, GEO, and developer SEO