llms.txt Complete Setup Guide: Help AI Assistants Understand Your Website

llms.txt is an emerging web standard that helps AI assistants like ChatGPT, Claude, and Perplexity understand your website's structure and purpose. Think of it as robots.txt for AI — but instead of restricting access, it provides context.

What Is llms.txt?

llms.txt is a plain text file (written in markdown) placed at the root of your domain: https://yoursite.com/llms.txt. When an AI assistant encounters your site, it reads this file to understand:

• What your site is about
• Which pages are most important
• Where to find documentation
• How to describe your product or service

Why Your Site Needs One

AI assistants make thousands of decisions about which sites to cite and how to describe them. Without an llms.txt file, they're guessing based on whatever content they can crawl. With one, you're providing authoritative context that shapes how AI represents your brand.

Run the llms.txt Checker to see if your current file exists and how it scores.

Template Structure

# Your Site Name

> A one-sentence description of what your site does.

## Key Pages

- [Homepage](https://yoursite.com): Brief description
- [Product](https://yoursite.com/product): What it does
- [Docs](https://yoursite.com/docs): API and developer docs
- [Pricing](https://yoursite.com/pricing): Plans and features
- [Blog](https://yoursite.com/blog): Articles and guides

## About

Your organization name and what you do. Include founding year
and key differentiators.

## Contact

- Website: https://yoursite.com
- Support: support@yoursite.com

Best Practices

1. Keep it concise — AI models work better with clear, dense information. Aim for 50-200 lines.
2. Use markdown formatting — Headings, lists, and links help AI parse structure.
3. List your 5-10 most important pages — Don't dump your entire sitemap. Curate.
4. Include a clear site description — One sentence that AI can quote directly.
5. Update quarterly — As your site evolves, keep the file current.
6. Don't block it — Make sure robots.txt allows access to /llms.txt.

Validation

After creating your file, validate it with the llms.txt Checker. It scores your file on:

• File existence and accessibility
• Proper markdown heading structure
• Site description clarity
• Key page links and descriptions
• Overall content quality

Also run the AI Readiness Check for a broader view of your site's AI search readiness.

Common Mistakes

• Too long — Don't include hundreds of URLs. AI assistants need curated context, not a data dump.
• No description — The site description is the most important element. AI models use it verbatim.
• Blocked by robots.txt — Some aggressive robots.txt rules accidentally block /llms.txt.
• Stale content — An outdated llms.txt is worse than none. Keep it synced with your site.