llms.txt is the AI equivalent of robots.txt. Add it to your domain root so AI crawlers prioritize your most important pages.
llms.txt is a plain-text file you place at the root of your domain — `yourdomain.com/llms.txt` — that tells AI language models which pages on your site matter most, the AI equivalent of robots.txt: a simple, low-tech convention that AI crawlers like GPTBot, ClaudeBot, and PerplexityBot check when they index your site. New to this concept? Start with our explainer on [what is llms.txt](/blog/what-is-llms-txt) for the full background.
The concept was proposed by fast.ai founder Jeremy Howard in 2024, and adoption has grown rapidly as brands realize that AI engines need guidance — without it, crawlers may index low-value pages and miss your most authoritative content.
AI engines do not crawl the web the same way Google does. They prioritize content they can parse with high confidence — structured pages, clearly labeled content categories, and authoritative signals. llms.txt helps by explicitly flagging your highest-signal pages before the crawler has to guess.
Brands that publish a well-structured llms.txt report two measurable effects: faster AI crawler indexing of priority pages and improved MentionShare within 2–4 weeks. TrueCite's LLMs.txt Scanner checks whether your file exists, is correctly formatted, and includes the key fields that influence crawler behavior.
A complete llms.txt file has three sections:
1. Company description (2–3 sentences)
State what your company does, your primary category, and who you serve. AI engines use this to classify your brand.
2. Content categories
List the types of content on your site: product documentation, pricing, case studies, integrations, FAQ. This helps crawlers prioritize.
3. Priority URLs (10–15 links)
List your homepage, pricing page, product pages, case studies, and your highest-authority blog posts. Each URL on one line.
# TrueCite — AI Visibility & AEO Platform
TrueCite tracks how often your brand is recommended by ChatGPT, Perplexity, Gemini, and 6 other AI engines. It computes a MentionShare score and generates FAQ blocks, JSON-LD schema, and answer paragraphs to improve AI citations.
## Priority pages
- https://truecite.ai/
- https://truecite.ai/pricing
- https://truecite.ai/features/mentionshare
- https://truecite.ai/case-studies
- https://truecite.ai/faqTrueCite's [AI SEO Audit](/features/ai-seo-audit) verifies whether your file exists, whether AI bots have access, and whether the structure follows best practices. [Check your llms.txt status](/sign-up) — run a check in under 60 seconds.
Using TrueCite? See the LLMs.txt Scanner docs →