Your SERP API export may list google.com as the ranked domain

Post from r/TechSEO: domain field said google.com
Post: r/TechSEO, u/DrKJING

Summary

A practitioner's Oct 30 sample of DataForSEO SERP API data returned most organic results with google.com in the domain field instead of the site that ranked, and no API error to flag it.

Search the raw responses for google.com/goto before trusting any domain in data pulled since Oct 22.

Most organic results in a sample of DataForSEO SERP API data taken on Oct 30 came back with google.com in the domain field where the ranked site should be, according to a measurement posted in r/TechSEO by u/DrKJING, a practitioner who tracks rankings with their own tool on that data. DataForSEO sells raw Google results by API. In the affected results the URL field holds a google.com/goto wrapper instead of the site’s address.

The wrappers are new in that account’s data. DrKJING re-fetched every Google organic task completed between Oct 15 and 29 and ran a dedicated sample on Oct 30:

Dates (UTC)Organic resultsWrapped as google.com/goto
Oct 15 to 2171,5310
Oct 22 to 2863,78299
Oct 2912,241355
Oct 30 sample4,3442,552

The earlier rows come from different keywords and depths than the Oct 30 sample, so the trend matters more than the exact rates. A string check run against a two-week-old export therefore proves nothing about the data this week’s report is built from.

Nothing about a wrapped result arrives as a failure. The tasks returned status 20000, the rank, title and snippet are all present, and only the address is missing, so a tracker that identifies a site by the domain field reads it as absent from results it still ranks in. DrKJING noticed that before measuring anything: on Oct 30 a site showed as outside the top 10 for its own brand name with a google.com entry among the other results, and the next request, read from the breadcrumb field instead, put it at #1. The raw item for that site shows the split:

{"type": "organic", "rank_group": 4, "domain": "google.com",
 "url": "https://google.com/goto?url=CAESVwHrOzAV…",
 "breadcrumb": "https://evergrowsolution.de",
 "website_name": "evergrowsolution.de"}

Google has routed result links through google.com/goto?url=<token> since the summer, which leaves any SERP API to resolve each link before it can say which site ranked. DataForSEO’s support told DrKJING that Google now caps how many goto links from a single results page can be resolved in quick succession. After the first five or six, each further link hits an interstitial whose delay grows, up to 40 seconds and sometimes longer, and neither switching IPs nor waiting clears it. DataForSEO’s stated fix is that the standard queue now resolves every goto URL, that live and high-priority modes resolve about half, and that a free endpoint, appendix/goto_url_converter, converts up to 100 URLs per request.

The fix had not reached the data by the morning of Oct 31. The standard queue still returned 61% of organic results wrapped on a re-run of the German and US keywords. The converter did work in that check, resolving both a batch of the previous day’s links and a set of fresh ones, with every domain matching the breadcrumb.

Reading the host out of breadcrumb holds up under checking. The breadcrumb starts with the domain for 94.9% of wrapped results in the Oct 30 sample. Of 200 wrapped goto URLs requested without following the redirect, each returned a 302 whose first Location host matched the breadcrumb domain wherever there was one. The eleven with no domain there were YouTube, Facebook, Reddit and Instagram results, where the breadcrumb carries view or follower counts and website_name carries the platform and channel name.

There is no shortcut by site. Of the 76 domains that appeared at least five times in the sample, 67 came back wrapped in some results and direct in others, so an allowlist of affected domains will not catch it. The keywords were hand-picked across eight countries, so 58.7% describes that mix rather than a Google-wide rate, and only DataForSEO was measured.

Wrapped URLs are also not the same problem as Google swapping the results it serves to rank trackers. There the SERP carries results Google does not serve to users; here the results are the usual ones and only the URL field points at Google.

What to do

All of this applies to the raw API responses, since the wrapping is invisible in a tool’s own ranking view.

  • Search every Google organic task since Oct 22 for google.com/goto in the URL field, or a domain of google.com. A handful of wrapped results is enough to move an average position.
  • Where a result is wrapped, take the host from breadcrumb and allow non-ASCII characters, since German domains with umlauts come through there intact. For platform results with no domain in the breadcrumb, read website_name. The full target URL is not in the breadcrumb at all, which is what the converter endpoint is for.
  • Backfill by running the wrapped goto URLs through the converter rather than re-requesting through live or high-priority modes, which resolve only part of the links. The converter was the one path returning real domains in the Oct 31 check; the standard queue still wrapped most results.
  • Check the same query, page, country and device in Search Console before reporting any drop dated Oct 30 or later. A missing address is not a lost ranking, which also makes it different from Google swapping which URL of yours ranks.