Google says share links should stay out of Search by default

Google added a section called “Only index content meant for Search” to its guide on preventing user-generated spam on Oct. 26, telling any service whose users create shareable pages, generated outputs or interactive tools to apply the noindex rule or block crawling in robots.txt by default. Search Engine Journal’s piece reports that the Search Central changelog files the change under blocking indexing of link-shared content.

That is a default-deny position new to the guide. Pages made “for direct sharing, collaboration, or utility rather than public search discovery” now start out excluded, and the platform opens up what it wants found. Search Engine Journal compared the text against the version last updated Dec. 10, 2025, and found the guide’s scope widened too: platforms open to abuse now include “tools that let users publish shared pages, generated content, or web apps,” and spammers are described as creating a large number of “pages or sites” where the old wording said sites. The older advice to noindex posts from new accounts until they earn a reputation survives unchanged.

The two routes are not interchangeable

Google offers noindex and robots.txt as alternatives in the same sentence, and only one of them keeps a page out of the index. Google’s own robots.txt introduction says the file “is not a mechanism for keeping a web page out of Google”: a disallowed URL can still be indexed when other sites link to it, and Google can then show the URL and the anchor text of those links with no description attached. The noindex documentation closes the loop, requiring that the page not be blocked by robots.txt, because a crawler that never fetches the page never reads the rule.

So for share links the safer bet is noindex with crawling left open, and the choice is less even than the guide’s wording suggests. Google’s “Control what you share” page, updated Oct. 21, lists robots.txt as applying to images and video, and noindex as applying to all content types.

Search Engine Journal’s piece supplies a live case. On July 27 its writer checked the share path of Claude, Anthropic’s AI assistant, and found it both disallowed in robots.txt and returning an X-Robots-Tag header with the value “none,” which Google treats as equal to noindex plus nofollow. Those two protections do not add up to more protection. Googlebot obeying the disallow never requests the page, so the header is never read, and the piece says it could not determine what Google saw when it first found those links.

What Google wants platforms to watch

The same update tells platforms to verify the site or subdomain in Search Console so manual actions and security issues arrive by email, and to check the Page Indexing and Search performance reports for unexpected spikes in indexed pages, impressions or queries that do not match what the platform is for. A new monitoring bullet pushes the tracking upstream into the product itself: alert on spikes in newly published public pages, and on a small number of users generating a large share of indexable URLs. Platform-side counters are the better instrument for that, since Search Console can lag or mislead: this month one site’s impressions hit zero while live URL tests still passed.

What to do

• Inventory the URL patterns that exist for sharing rather than discovery: share links, public document views, generated-output permalinks, embeds, and preview or export URLs. Those are the paths the new section covers.
• For patterns already in the index, remove the robots.txt Disallow, serve X-Robots-Tag: noindex on those responses, and let Googlebot crawl them until the URLs drop out.
• For patterns Google has never crawled, a robots.txt block saves crawl but leaves the external-link hole open, and a noindex sitting behind that block does nothing. Share URLs get posted publicly, since that is what they are for, so the noindex-only setup is the one that holds.
• Check the Page Indexing report for “Indexed, though blocked by robots.txt.” Google’s help for that status already says to remove the block and use noindex, which is the same fix arriving from the other direction.
• Verify each subdomain that serves user content in Search Console separately. An unverified property gets no manual action email.