Public AI Access

llms.txt is not detected

Find why llms.txt is missing or unreadable, then align the file with robots policy, sitemap discovery, and useful public content paths.

Run llms.txt Checker Browse workflows

Workflow shape

1Confirm the symptom

2Collect live evidence

3Fix in the right order

4Retest the same URL

What this means

llms.txt is a context file. It helps most when it is reachable, concise, current, and aligned with crawler policy, sitemap URLs, and important public pages.

Use this when

The llms.txt checker cannot find your file.
The file exists locally but returns 404, redirects badly, or is blocked in production.
You want to make the file useful instead of only publishing a placeholder.

Evidence

What to collect before changing anything

Response status, content type, and final URL for /llms.txt.
Robots.txt directives that might block the file or listed pages.
Sitemap URLs and priority pages listed in llms.txt.
Last updated date and owner for keeping the file current.

Diagnosis

Run the checks in this order

1Run llms.txt Checker and verify the file returns 200 at the root path.
2Run robots.txt Analyzer or AI Crawler Audit to ensure policy is aligned.
3Run Sitemap Checker and compare the listed public pages with llms.txt.
4Run Public AI Access Check on pages referenced by the file.
5Remove stale or low-value links that do not help a reader understand the site.

Fix plan

What to fix first

1Publish /llms.txt at the root with a plain text response.
2List only high-value public pages with short, accurate descriptions.
3Keep private, account, report, and low-value generated routes out of the file.
4Update it when major hubs, docs, or product pages change.
5Retest after deployment and cache purge.

Retest

How to confirm the fix worked

Fetch /llms.txt directly and confirm 200, plain text, and readable content.
Run llms.txt Checker and Public AI Access Check again.
Confirm the file links resolve to indexable canonical pages.

FAQ

Common questions

Will llms.txt make AI tools cite my site?+

No. It can provide useful context, but citation depends on crawl access, authority, source clarity, and content usefulness.

Should every URL be listed in llms.txt?+

No. Keep it curated. A short list of important public pages is more useful than a copied sitemap.

Loading CheckWebs…