Use Google Search Central and WordPress documentation reviewed in July 2026 as a starting point. We considered obsolete directives, JavaScript and deletion or migration decisions. Search and CMS behavior change, so recheck official documentation and actual responses before publishing.
noindex tells supporting search engines not to show a URL in search results.It neither restricts access nor improves rankings. For noindex to work, search engines must be able to crawl the page and read it; inspect the public HTML or HTTP header after release.
Decide why the page should not appear before choosing noindex, canonical or 404
noindex may suit confirmation pages or internal search results that do not need search visibility. Use authentication for confidential content, a 301 for a move, 404/410 for deletion, and canonical signals for duplicates. Mixing objectives can cause unintended exclusion or missed removals.
| Objective | Tell supporting search engines not to show the target URL in search results |
|---|---|
| HTML implementation | <meta name="robots" content="noindex"> |
| Non-HTML implementation | Return the HTTP header X-Robots-Tag: noindex in the response |
| Important condition | Allow crawling so that search engines can read noindex |
| Timing | After recrawling and processing; removal is not necessarily immediate |
| How to verify | Compare the public response, HTTP headers, Search Console URL Inspection and target URL list |
noindex is a directive to exclude a page from search results
Think about publication in three stages: who can view a page, whether crawlers can fetch it, and whether it is indexed and shown in search. noindex primarily concerns the third stage.
Access
Whether a user can open the URL. Protect confidential or internal documents with login, Basic authentication and permissions.
Crawling
Whether a search crawler can fetch the URL. robots.txt primarily manages crawler access.
Indexing
Whether a fetched URL is listed in search results. noindex addresses this stage.
noindex alone is often enough
Google follows links by default, so to exclude a page from results, noindex is sufficient. You may writenoindex,follow to make the intent explicit, but addingfollow is not required.
none has the same meaning as noindex,nofollow — unless there is a clear reason not to follow links, avoid adding nofollow mechanically so that the operational intent stays clear.
noarchive、nocache、nositelinkssearchbox . To control search snippets or image previews, check current directives such asmax-snippet and max-image-preview for the intended purpose.How to decide which pages should use noindex
Bulk exclusion based on vague reasons such as thin content or SEO concerns may remove valuable search entry points. First explain in one sentence why the URL should be excluded and why it should remain publicly available with a 200 response.
Viewable by users but not needed as search entry points
- Purchase, application and inquiry confirmation pages
- Internal search results and filters with no search value
- Public previews and other pages temporarily shared externally
- PDFs and reference files that need no search visibility
- Supporting members-area pages visible before authentication
noindex does not achieve the objective
- Confidential or internal content: authentication, permissions and network restrictions
- Moved to another URL: 301 or 308 redirect
- Deleted with no replacement: 404 or 410
- Preferred URL among duplicates: canonical or redirect
- Low-quality content: improve, consolidate, delete or review information architecture
Making difficult page-level decisions
| Example | Default decision | Reason and notes |
|---|---|---|
| Thank-you page | noindex candidate | Direct access may be necessary, but search-entry value is usually low. Separately ensure benefits cannot be obtained by guessing the URL. |
| Internal search results | noindex candidate | When every search term creates many URLs, also review internal linking, parameter design and crawl load. |
| Staging environment | Prioritize authentication | noindex alone cannot prevent outside access. Use Basic authentication, IP restrictions or a VPN as the primary control. |
| Duplicate product and article URLs | Consider canonical | Use canonical or a redirect to consolidate signals toward a preferred URL. noindex may suit pages not needed in search; keep the objectives separate. |
| Expired campaign | Choose by situation | Consider a redirect when a successor exists, update and publish useful material, or use 404/410 when no alternative exists. |
| Tag and category listings | Do not exclude in bulk | Review search demand and internal navigation value. Compare adding context, consolidation or structural changes for thin listings. |
How to configure noindex correctly
Use a robots meta tag in the head for HTML pages and X-Robots-Tag for non-HTML files such as PDFs and images. Check the public response after CDN and caching rather than relying only on the CMS settings screen.
HTML pages: robots meta tag
<head>
<meta name="robots" content="noindex">
</head>
Values are generally case-insensitive, but consistent spelling makes audits easier. Consolidate overlapping robots meta tags and plugin directives where practical.
PDFs, images and generated files: X-Robots-Tag
HTTP/2 200
Content-Type: application/pdf
X-Robots-Tag: noindex
X-Robots-Tag can also be used for HTML. Decide whether the application, web server or CDN adds it, and check the final response after redirects.
Go example
func preview(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Robots-Tag", "noindex")
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// Return the response body
}
Nginx example
location = /files/internal-guide.pdf {
add_header X-Robots-Tag "noindex" always;
}
WordPress: remember to remove site-wide settings
Discouraging search engine indexing is a site-wide setting
The search engine visibility option under WordPress Settings → Reading emits a site-wide noindex directive. If used during development, record the launch date, person responsible for removal and URLs to check.
Page-level settings may be available through an SEO plugin or theme. Inspect final HTML and response headers for overlapping plugin, theme and server directives.
Cautions when adding or removing noindex with JavaScript
Google may recognize noindex added to meta robots after rendering. Avoid removing noindex from the initial HTML with JavaScript: Google may skip rendering after seeing the initial directive, so the removal may not run.
noindex, robots.txt, canonical and 404 compared
All affect search engines, but solve different problems. Decide the objective and HTTP response first and avoid conflicting signals.
| Method | Main purpose | Search results | Suitable for | Cautions |
|---|---|---|---|---|
| noindex | Exclude the URL from search results | Excluded after recrawling | Must remain viewable but is not needed as a search entry point | The page must be crawlable for the directive to be read |
| robots.txt | Restrict crawler fetching | The URL alone may still appear | Manage crawl load or crawling of specific paths | Does not reliably remove a page from search results |
| canonical | Indicate a preferred URL among duplicates | Encourage consolidation to the preferred URL | Organize identical or similar URLs | A signal, not a directive; not a substitute for noindex |
| 301 / 308 | Permanent move | Encourage replacement with the destination | URL changes, consolidation and successor pages | Avoid bulk redirects to unrelated pages |
| 404 / 410 | Indicate that content no longer exists | Removed after processing | Removed pages without a replacement | Update important internal links and sitemaps too |
| Authentication and permissions | Restrict access itself | Indexing is difficult when fetching is blocked | Confidential information, members-only content and development environments | Security design, not an SEO measure |
Do not combine robots.txt blocking and noindex for the same URL
If robots.txt blocks crawling, Google cannot see the page's noindex directive. Keep URLs crawlable when returning noindex for search exclusion. If reducing crawling is also a goal, first verify removal from the index, then review URL design, internal links and parameter management separately.
Do not combine canonical and noindex merely as insurance
canonical signals a preferred URL, while noindex asks for a URL to be excluded from search. Adding both without a clear intent makes the desired outcome ambiguous. For consolidation, align canonical signals, internal links and sitemaps.
Include canonical URLs intended for search in XML sitemaps
Leaving noindex URLs in a sitemap creates a conflict between URLs presented for search and a directive to exclude them. Review automatic generation rules and generally include only canonical URLs intended for search.
Six steps to verify noindex
Do not rely on an admin screen or a fragment of source code. Review the URL inventory, public response and Google-side status separately. Check www variants, HTTP/HTTPS, trailing slashes, parameters and localized URLs for different responses.
Confirm target URLs and reasons
Record URLs, patterns, templates, reasons, configuration dates, removal deadlines, owners and alternatives. For broad regex or CMS rules, define URLs that should remain searchable as exceptions first.
Inspect final HTML in the browser
View Source and the developer tools Elements panel can differ. Inspect both initial server HTML and the DOM after JavaScript runs.
Fetch HTTP headers and status
Check whether the response is 200, 3xx or 4xx, whether X-Robots-Tag remains at the redirect destination, and whether the CDN serves stale values.
Check for robots.txt blocks
For URLs Google must read, check robots.txt, WAF, IP restrictions, mandatory login and geographic restrictions that might prevent fetching.
Use Search Console URL Inspection
Review indexed information and the live URL test separately. Google's last fetch and the current response may differ in time.
Check again after allowing time
noindex may not take effect immediately. Infrequently crawled URLs can take longer. Record configuration, recrawl request and inspection dates to distinguish errors from processing delays.
Check the public response from the command line
# Fetch HTML after redirects and look for meta robots
curl -sL https://example.com/page/ | grep -i "robots"
# Follow redirects and inspect the final response headers
curl -sIL https://example.com/file.pdf
# Check X-Robots-Tag only
curl -sIL https://example.com/file.pdf | grep -i "x-robots-tag"
site:Do not rely on search alone:Search operator results are not a complete index listing and can lag behind. Use the public response and Search Console URL Inspection as primary checks.Why noindex is not applied or removed
When a page remains in search despite a tag, or fails to return after removal, check the response for each URL and each delivery stage in order.
robots.txt or authentication blocks crawling
Google cannot fetch the page to read the addition or removal. Keep it crawlable for search exclusion, but retain authentication when protecting confidential content.
Meta tags and HTTP headers conflict
Even if noindex is removed from HTML, the CDN or server may still return X-Robots-Tag. Check both.
CDN or page cache is stale
The origin may be updated while edge caches still return old HTML or headers. Clear the cache and fetch externally again.
A different URL is being inspected
The indexed and inspected URLs may differ by www, trailing slash, parameters, AMP, print version or locale.
JavaScript removes noindex
Removing noindex from initial HTML with JavaScript may not be recognized. Pages intended for indexing should not return noindex in the initial response.
Waiting for Google to recrawl
Google may retain an older state even when implementation is correct. Compare the live test with indexed information and check again later.
When you need to hide a result quickly
The Search Console removals tool temporarily hides URLs from Google search results. It can help in urgent situations, but is not permanent. Also establish the URL's final state using noindex, authentication, 404/410 or a redirect as appropriate.
Search Console removals tool
Use it to hide search results quickly after accidental publication or exposure of confidential information. Choose the URL or prefix carefully and review the impact.
Address the URL itself
Use noindex for public pages that should not appear in search, 404/410 for removal, redirects for moves and authentication for confidential content. The intended state must remain after the temporary removal expires.
Steps to remove noindex and allow search visibility again
Forgetting to remove noindex at launch can severely affect site-wide traffic. Include crawling, canonical URLs, sitemaps and internal links in the publication procedure as well as tag removal.
Remove meta tags and X-Robots-Tag
Check HTML, the application, web server, CDN and SEO plugins, and remove every noindex directive from the target URL.
Verify a 200 OK and public content
Ensure the URL intended for indexing returns a normal 200 response and its content is available without login or geographic conditions.
Review robots.txt and canonical URLs
Check for rules blocking crawling and whether canonical points to another URL.
Restore the sitemap and internal links
Include canonical URLs intended for search in the XML sitemap and add normal internal links from important pages.
Inspect the live URL
Check the current response with the Search Console live test and request indexing if appropriate.
Monitor reindexing
Search visibility does not return on a guaranteed schedule. Review URL Inspection, sitemaps and search performance over a defined period.
noindex checklist before publication or removal
Treat the setting as change management rather than a one-off task to reduce accidental broad application and missed removals.
Official sources
Search specifications change. Before implementation, check these primary sources and official documentation for your CMS, CDN and hosting provider.
- Google Search Central: Block search indexing with noindex
- Google Search Central: Introduction to robots.txt
- Google Search Central: Robots meta tag and X-Robots-Tag specifications
- Google Search Central: Consolidate duplicate URLs
- Google Search Central: Build and submit a sitemap
- Google Search Central: JavaScript SEO basics
- Google Search Console: Removals tool
- WordPress.org|Reading Settings
This article does not guarantee rankings or traffic improvements for any site. Processing time and selected signals depend on URL state, crawl frequency, site structure and other factors.
Frequently asked questions about noindex
Does setting noindex immediately remove a page from search results?
Not necessarily. Search engines must recrawl the URL and process noindex. Check the public response and Search Console URL Inspection separately.
Can noindex and robots.txt be used together?
If robots.txt blocks a URL, Google cannot fetch its noindex directive. Keep the URL crawlable and return noindex when the goal is exclusion from search.
What is the difference between noindex,follow and noindex alone?
Google follows links by default, so noindex alone is sufficient when links may be followed. noindex,follow makes the intent explicit but follow is not required.
Should I use canonical or noindex?
To indicate a preferred URL among duplicates, consider canonical or a redirect. To exclude the URL itself from search, consider noindex. A canonical signal does not guarantee the URL Google selects. Decide based on the objective and whether an alternative exists.
Is WordPress's setting to discourage search engines from indexing the site safe?
It affects the whole site, so avoid leaving it enabled on production by mistake. It does not restrict access; use authentication or access controls for staging.
Can a PDF be excluded from search results?
Yes. For PDFs, images and other non-HTML resources, return X-Robots-Tag: noindex in the HTTP response header.
Should deleted pages use noindex?
Use a 301 redirect when a suitable replacement exists, or a 404 or 410 when the page has been removed with no replacement. There is no need to retain an unnecessary 200 page just to return noindex.
A page has not returned to search after removing noindex. What should I check?
Check both meta tags and X-Robots-Tag, robots.txt, HTTP status, canonical, CDN cache and URL variants. Then inspect the live URL in Search Console and request recrawling.
Conclusion: manage scope, reason, verification and removal together
noindex is useful for URLs that do not need search visibility. It serves a different purpose from access control, crawl control, duplicate consolidation, redirects and deletion.
- Limit it to URLs that must remain viewable but are not needed as search entry points
- Use meta robots for HTML and X-Robots-Tag for non-HTML resources
- Do not block URLs with robots.txt when crawlers need to read noindex
- Distinguish its purpose from canonical, 301, 404 and authentication
- Check public responses and Search Console in stages
- Record deadlines, owners and removal steps
To prevent noindex mistakes, check consistency across the publication path, including templates, CMS, CDN, sitemap and internal links, rather than only page-level settings.