llms.txt: The Complete Implementation Guide for 2026

What llms.txt is and why it matters

llms.txt is a plain text file you place at the root of your domain — `yourcompany.com/llms.txt` — that tells AI language models which pages on your site are most useful for their purposes and which should be excluded.

Think of it as robots.txt for AI engines, but more informative: instead of just allowing or blocking crawlers, it tells them *which pages matter* and *why*. An AI engine crawling your site with llms.txt can immediately identify your most valuable content without having to discover and evaluate every page.

Having a well-configured llms.txt signals to AI engines that your site is AEO-aware, structured for machine readability, and worth prioritizing when building their knowledge representations. For a broader introduction to AI crawler management, read the [LLMs.txt Guide](/blog/llms-txt-guide).

The difference between llms.txt and llms-full.txt

llms.txt: An index file at the root of your domain. It lists your key pages with brief descriptions — what each page contains and why an AI engine might want to use it. Lightweight, easy to create and maintain.

llms-full.txt: A concatenation of the full text content of your most important pages. Intended for AI engines that want to consume your entire site's content in a single request. More powerful but requires more maintenance — update it whenever key pages change.

Start with llms.txt. Add llms-full.txt once llms.txt is live and you've confirmed AI crawlers are accessing it.

llms.txt file format — complete working example

# Your Company Name
> Your Company is a [brief category description] that helps [ICP] [solve problem].

## Key pages

- [Homepage](https://yourcompany.com): Company overview, product description, and key use cases for [ICP].
- [Product Overview](https://yourcompany.com/product): Full product feature list, integration capabilities, and pricing tiers.
- [Pricing](https://yourcompany.com/pricing): Pricing plans from $X/month. Includes feature comparison table.
- [Features: [Feature Name]](https://yourcompany.com/features/feature-name): How [feature] works, use cases, and implementation guide.
- [Use Cases](https://yourcompany.com/use-cases): How [ICP 1], [ICP 2], and [ICP 3] use [your product].
- [FAQ](https://yourcompany.com/faq): Frequently asked questions about [product], pricing, and implementation.
- [Case Studies](https://yourcompany.com/case-studies): Real customer results including [specific metrics].
- [Blog: [High-value post title]](https://yourcompany.com/blog/post-slug): [Brief description of what this post covers].

## Excluded
- [Terms of Service](https://yourcompany.com/terms): Legal content, not useful for AI consumption.
- [Privacy Policy](https://yourcompany.com/privacy): Legal content, not useful for AI consumption.
- [Dashboard](https://yourcompany.com/dashboard): Authenticated application, not public content.

What to include in llms.txt

Include:

• ▸Homepage with a clear company description
• ▸Product and feature pages — these are the most cited by AI engines
• ▸Pricing page — buyers ask about pricing constantly; AI engines need this
• ▸Use case pages — highly specific content about who benefits and how
• ▸High-value blog posts (comprehensive guides, how-tos, category explainers)
• ▸Case studies with quantified results
• ▸FAQ page

Exclude:

• ▸Legal pages (terms, privacy policy) — not useful for AI consumption
• ▸Authenticated/dashboard pages — AI engines can't access these
• ▸Low-quality or thin content pages
• ▸Duplicate content pages
• ▸Admin and internal pages

Creating llms-full.txt

llms-full.txt is your most important pages concatenated as plain text. The format:

1. 1.For each page: start with a `# Page Title` heading and the URL
2. 2.Include the full text content — no HTML, no navigation menus, no boilerplate
3. 3.Use section headers (`##`, `###`) for clarity within each page
4. 4.Update when key pages change (monthly is sufficient for most sites)

Most teams start with 5–10 pages in llms-full.txt: homepage, product overview, top 3 feature pages, FAQ, and top 2 case studies. Quality over quantity — 8 excellent pages outperform 50 thin pages.

Checking AI crawler access in robots.txt

llms.txt is only useful if AI crawlers can access your site. Check your robots.txt for these common crawler user agents:

• ▸`GPTBot` — OpenAI (ChatGPT)
• ▸`PerplexityBot` — Perplexity
• ▸`ClaudeBot` / `anthropic-ai` — Anthropic (Claude)
• ▸`Google-Extended` — Google (Gemini training data)
• ▸`Googlebot` — Google web crawling (separate from AI training)
• ▸`Bytespider` — ByteDance

If any of these are blocked in your `User-agent: BotName / Disallow: /` directives, those AI engines cannot crawl your site — and won't cite your content. Use TrueCite's LLMs.txt Scanner to check your crawler access status for all major AI bots.

Monitoring your implementation

After creating llms.txt and verifying crawler access in robots.txt, check your implementation in TrueCite's [AI SEO Audit](/features/ai-seo-audit). The scanner verifies: (1) llms.txt exists and is accessible, (2) llms-full.txt exists (if applicable), (3) sitemap.xml is accessible, (4) each major AI crawler is explicitly allowed.

Run a new MentionShare scan 2–4 weeks after implementing llms.txt. Improvement in your MentionShare across Perplexity (live web crawler) is the fastest feedback loop — llms.txt changes can show results within weeks.

[Check your AI crawler readiness](/sign-up) — TrueCite's LLMs.txt Scanner shows your implementation score and exact fixes in minutes.