llms.txt is the AI equivalent of robots.txt. Add it to your domain root and AI crawlers will prioritize your most important pages. Here is exactly what to put in it.
llms.txt is a plain-text file you place at the root of your domain — `yourdomain.com/llms.txt` — that tells AI language models which pages on your site matter most. It is the AI equivalent of robots.txt: a simple, low-tech convention that AI crawlers like GPTBot, ClaudeBot, and PerplexityBot check when they index your site.
The concept was proposed by fast.ai founder Jeremy Howard in 2024, and adoption has grown rapidly as brands realize that AI engines need guidance — without it, crawlers may index low-value pages and miss your most authoritative content.
AI engines do not crawl the web the same way Google does. They prioritize content they can parse with high confidence — structured pages, clearly labeled content categories, and authoritative signals. llms.txt helps by explicitly flagging your highest-signal pages before the crawler has to guess.
Brands that publish a well-structured llms.txt report two measurable effects: faster AI crawler indexing of priority pages and improved MentionShare within 2–4 weeks. TrueCite's LLMs.txt Scanner checks whether your file exists, is correctly formatted, and includes the key fields that influence crawler behavior.
A complete llms.txt file has three sections:
1. Company description (2–3 sentences)
State what your company does, your primary category, and who you serve. AI engines use this to classify your brand.
2. Content categories
List the types of content on your site: product documentation, pricing, case studies, integrations, FAQ. This helps crawlers prioritize.
3. Priority URLs (10–15 links)
List your homepage, pricing page, product pages, case studies, and your highest-authority blog posts. Each URL on one line.
# TrueCite — AI Visibility & AEO Platform
TrueCite tracks how often your brand is recommended by ChatGPT, Perplexity, Gemini, and 6 other AI engines. It computes a MentionShare score and generates FAQ blocks, JSON-LD schema, and answer paragraphs to improve AI citations.
## Priority pages
- https://truecite.ai/
- https://truecite.ai/pricing
- https://truecite.ai/features/mentionshare
- https://truecite.ai/case-studies
- https://truecite.ai/faqTrueCite's LLMs.txt Scanner verifies whether your file exists, whether AI bots have access, and whether the structure follows best practices. Run a check in under 60 seconds — no account required for the first scan.
Sukanta Mohapatra
Founder & CEO, TrueCite — AEO Strategist
B2B SaaS builder who noticed great products were invisible to AI engines. Built TrueCite to fix that. About TrueCite →