Crawl and Indexing

Noindex URL appears in the sitemap

Resolve the mixed signal created when a URL is listed in XML sitemap but also marked noindex in HTML or HTTP headers.

Run Sitemap Checker Browse workflows

Workflow shape

1Confirm the symptom

2Collect live evidence

3Fix in the right order

4Retest the same URL

What this means

A sitemap should highlight canonical URLs you want crawled and indexed. Listing noindex URLs wastes crawl attention and makes your URL policy harder to trust.

Use this when

A crawler reports noindex pages inside sitemap.xml.
Search Console says submitted URL marked noindex.
You are cleaning index coverage before submitting a sitemap again.

Evidence

What to collect before changing anything

The sitemap URL and the exact conflicting page URL.
Meta robots and X-Robots-Tag headers from the live response.
Canonical URL, final redirected URL, and HTTP status.
The template or data source that generated the sitemap entry.

Diagnosis

Run the checks in this order

1Run Sitemap Checker and export the conflicting URL list.
2Run SEO Analyzer or AI Meta Tag Validator on a sample URL to confirm noindex source.
3Run Canonical URL Checker to verify whether the sitemap URL matches the preferred URL.
4Check whether the URL is intentionally noindex or accidentally inherited from a layout.
5Trace the sitemap generator logic to the content type or route group.

Fix plan

What to fix first

1If the page should rank, remove noindex and keep it in the sitemap.
2If the page should not rank, keep noindex and remove it from the sitemap.
3Remove redirected, 404, duplicate, and parameter-heavy URLs from sitemap output.
4Keep lastmod truthful and update only when visible content or index policy changes.
5Resubmit the cleaned sitemap after deployment.

Retest

How to confirm the fix worked

Re-run Sitemap Checker and confirm no noindex URLs remain.
Fetch a sample URL and confirm the robots directive matches the sitemap decision.
Check Search Console submitted URL reports after the next crawl.

FAQ

Common questions

Can noindex pages stay in the sitemap for discovery?+

They should usually be removed. Sitemaps are best used for canonical URLs you want indexed.

Is robots.txt enough to hide a URL from the sitemap?+

No. robots.txt controls crawling, while sitemap generation is a separate URL inventory decision.

Loading CheckWebs…