Reverting a bad noindex did not bring 85,000 pages back
Summary
A large database site still has one page indexed in Search Console months after the noindex rule that emptied it was reverted, and Googlebot keeps refetching the rest without indexing them.
Before calling it a quality problem, check Crawl stats by Googlebot type and confirm the fetches come from smartphone Googlebot, since only those update the index.
A VTuber database that had about 85,000 pages indexed in Search Console now has one. Kaz, a solo developer who built the free profile database Bācharu, posted the full timeline to r/TechSEO: a rule meant to put noindex on thin small-creator pages was set far too strict and applied it to most of the site. The rule went live on 21 April, was reverted on 21 May, and six months on the indexed count has not recovered.
At the April peak the site was getting around 6,600 impressions a day. The single indexed URL today is the homepage, which still ranks about third for “vtuber database”. Most of the rest sits in two Search Console buckets: 76,849 URLs at “Crawled - currently not indexed” and 36,620 at “Discovered - currently not indexed”.
The first explanation offered in the thread was that Google had cut the crawl rate for the whole site after most of what it fetched came back noindex. Crawl stats did not support that. Kaz reported roughly 9,600 Googlebot requests over the last 90 days on the new domain. That is about 100 a day, and 99% of them were logged as refresh rather than discovery. That is thin for a site this size, but it is not a site Google has stopped visiting, and it moves the question off crawl budget, which is rarely the cause of a large “Crawled - currently not indexed” pile.
Two Search Console reports then disagree about whether the pages are being crawled at all. Kaz says the crawl stats examples show Googlebot fetching profile pages as recently as 1 November, around 1,200 different profiles in that window. URL Inspection on those same URLs returns a last crawl date in April and the same not-indexed verdict. A site that is being refetched and declined is in a different position from one that is being ignored, and the two call for different work.
Two ordinary explanations come before any quality verdict. In Crawl stats, “By Googlebot type” shows whether those profile fetches came from smartphone Googlebot or from page resource loads, and only the first updates the index. The exact URL strings in the crawl examples also need comparing against the ones being inspected, including trailing slash, case, parameters and which domain they sit on. Stale reporting could account for part of that April date too, since Google has said indexing changes surface in performance reports before the indexing report catches up.
If the fetches are smartphone Googlebot and the URLs match, the likelier explanation is a priority call rather than a technical block. The pages return 200, are robots-allowed and self-canonical, carry no noindex in the header or the meta, and render server-side, with around 70,000 characters of text in the raw HTML on a large profile. Google is fetching those pages, reading them, and still not giving them a slot. On 100,000 templated profiles merged from fan wikis and platform data, the most ordinary reason is that most of them look too similar to each other and too close to the sources they aggregate.
An August finding in the thread fits that reading from the other side. The profile pages turned out to be close to orphaned, reachable from the sitemap and nothing else, with zero referring URLs. A sitemap entry is a request to index a page, and an internal link is the part of that request that carries weight. Browse pagination, an A-Z index and 30 related-profile links per page went in afterwards, which is the right repair and a slow one.
What to do
Stop asking for 100,000 index slots. Pick 100 to 200 profiles with genuinely unique data, strengthen them, and link them from hub pages that the homepage links to, which is where u/FilippeTraum, the commenter who worked through the crawl numbers, landed as well. Kaz is already running that test with 300 profiles and a Featured hub. Follower history and stats are the fields Kaz says the wikis do not carry, so a strengthened profile should lead with those rather than with the merged biography. Leaving the near-empty profiles at noindex on purpose is defensible for the same reason, in that it keeps the good pages from being assessed alongside them.
The Indexing API is not a way out. Kaz pinged about 300 profiles through it, every request was accepted, and none produced a crawl; that endpoint only supports JobPosting and BroadcastEvent pages. Manual indexing requests worked mechanically, in that the Featured hub was crawled the same day with the right canonical picked, and the verdict still came back “Crawled - currently not indexed”.
The domain move also has a loose end. Search Console still picks the old .xyz URL as canonical for some inner pages even though they 301 to .io. The old domain was briefly removed around 20 October, about seven months after the move. The redirects went back up on 22 October with a Change of Address filed. On a move of this size the per-URL 301s belong in place for years rather than months, and they stay until the old URLs stop turning up as the chosen canonical.