Google ranked a page body that stayed hidden until JavaScript ran

A game discovery site with roughly 960,000 programmatic pages kept the entire body of every page inside a <div hidden> until JavaScript ran, and Google indexed and ranked those pages regardless. The owner of IndieDex.gg posted the Google Search Console numbers to r/TechSEO on October 11, covering the nine weeks that markup was live. Seventy-two game pages took clicks in the first half of September. One of them averaged position 9.2 with its whole body inside the hidden div.

Putting the wrapper back turned out to be the riskier move. The owner had removed the hidden div out of caution, and when they tested restoring it, the change pushed the title, canonical and robots tags out of <head> on every page. The likeliest explanation is parse order: a <div> is not valid inside <head>, so the parser closes the head where the wrapper opens, and everything after it, metadata included, lands in the body.

Those two problems are not the same size. Google rendered its way past the hidden body, which the thread’s own click data shows. A canonical or robots tag outside <head> is safest assumed to do nothing, and on this site most URLs are held out of the index by a programmatic noindex. A directive covering the majority of 960,000 pages would have stopped applying, and nothing in Search Console announces that a tag moved.

Unhiding the body was still worth doing, for a different audience than the one that prompted it. The thread cites Vercel’s analysis finding that none of the major AI crawlers execute JavaScript, and extraction libraries such as Mozilla Readability discard hidden elements outright. Content that only appears after JavaScript therefore reaches neither the training crawlers (GPTBot, ClaudeBot) nor the retrieval fetchers that assemble answers (OAI-SearchBot, PerplexityBot, Claude-SearchBot). The retrieval group is the one that decides citations. OpenAI’s bot documentation draws that line explicitly: sites opted out of OAI-SearchBot are not shown in ChatGPT search answers, while disallowing GPTBot only signals that content should not be used to train models.

The site’s answer was Next.js htmlLimitedBots, which sends matched user agents blocking metadata instead of streamed metadata, so the head tags arrive in the initial HTML response rather than later in the stream. One detail in that doc is easy to miss. A custom value overrides the Next.js default list rather than extending it, and that default covers Google’s crawlers, Bingbot, Twitterbot and Slackbot, so a regex naming only AI user agents drops Bingbot out of blocking metadata.

What to do

• Compare the rendered head before and after any rendering change, per template rather than on one URL. Confirm that title, canonical and meta robots sit inside <head> and above any body markup. Raw-HTML checks pass markup that the rendered DOM breaks, the same gap that lets hreflang checkers approve tags AI crawlers never see.
• Decide the hidden-body question on the non-rendering fetchers, not on Googlebot. A template whose content appears only after JavaScript is absent from AI answers no matter how well it ranks.
• Ship rendering changes template by template. Sitebulb’s ecommerce rendering guide makes the same split, moving category listings into HTML while leaving checkout client-side, which keeps a head-tag regression off every URL at once.
• If you set htmlLimitedBots, carry the default entries into your own regex alongside the AI user agents.