The Complete Guide to llms.txt for AI Search

As AI chatbots become a primary way people discover information, website owners need a new tool to communicate with these models. Enter llms.txt — a simple text file that tells AI language models what your site is about.

What Is llms.txt?

llms.txt is a plain text file placed at the root of your website (e.g., example.com/llms.txt). It provides a structured overview of your site's content, designed specifically for LLMs to consume.

Think of it as:

• robots.txt → tells crawlers what they can't access
• sitemap.xml → tells search engines what pages exist
• llms.txt → tells AI models what your content means

Why Do You Need It?

AI models like ChatGPT and Claude crawl the web to build their knowledge. When they encounter your site, llms.txt gives them:

1. A summary of what your site offers — so they understand your brand
2. Links to key pages — so they can find your most important content
3. Context about your business — so they can accurately represent you in responses

Without it, AI models have to guess your site's purpose by scraping random pages.

How to Create an llms.txt File

Basic structure

# Site Name
> A brief description of what your site does

## About
- [About Us](https://example.com/about)
- [Contact](https://example.com/contact)

## Documentation
- [Getting Started](https://example.com/docs/getting-started)
- [API Reference](https://example.com/docs/api)
- [Tutorials](https://example.com/docs/tutorials)

Best practices

1. Keep it concise — AI models process text efficiently, but shorter is still better
2. Link to your most important pages — prioritize quality over quantity
3. Use descriptive link text — help models understand what each page is about
4. Update it regularly — when you add major content, update your llms.txt
5. Place it at the root — /llms.txt, not /docs/llms.txt

Advanced: llms-full.txt

For sites with extensive documentation, you can create a llms-full.txt that contains the full text of all key documents. Link to it from your main llms.txt:

# My Documentation
> Complete developer documentation

- [Full Documentation](/llms-full.txt)

Real Examples

Stripe

# Stripe
> Financial infrastructure for the internet

## Products
- [Payments](https://stripe.com/payments)
- [Billing](https://stripe.com/billing)
- [Connect](https://stripe.com/connect)

GitHub

# GitHub
> The world's leading AI-powered developer platform

## Docs
- [Documentation](https://docs.github.com)
- [REST API](https://docs.github.com/rest)
- [GraphQL API](https://docs.github.com/graphql)

Create Yours in Seconds

Writing an llms.txt file manually can be tedious. Our free llms.txt generator can create one for you automatically:

1. Enter your website URL
2. We crawl your key pages
3. Generate a structured llms.txt file
4. Download and deploy it to your site

Common Mistakes to Avoid

Conclusion

llms.txt is a simple but powerful tool for AI search optimization. As AI continues to reshape how people discover information, having this file gives you a competitive edge.

Generate your llms.txt now →