A 200 while your site is down can turn pages into duplicates
Summary
Google's John Mueller says a downtime page served with a 200 can be folded in as a duplicate of other sites showing the same maintenance text, and because duplicates get crawled less often, the damage outlasts the outage.
Serve a 503 with Retry-After during downtime, and set it at the CDN or load balancer so it still answers when the application itself is down.
Google’s John Mueller named two costs of returning a 200 status code with an error body while a site is down, in a LinkedIn reply reported by Search Engine Roundtable: the “try again later” text gets indexed, and pages carrying that text can be eliminated as duplicates of other sites showing the same message. The second cost outlasts the outage.
The question put to Mueller was whether duration changes the answer. Ty Holland, a commenter on the post, asked whether a 200 with a back-soon message causes problems if it lasts a few hours, or only during longer outages. Mueller’s reply, as Search Engine Roundtable quotes it, splits the two. The indexed maintenance text “will fix itself automatically once the content is back”, while the duplicate problem “can take quite some time to get ‘un-deduplicated’ because duplicates tend not to get crawled as often”.
A few hours of indexed maintenance text is cosmetic, and it reverses on the next crawl. The duplicate half is a crawl-frequency problem, and crawl frequency is the part nobody can hurry. Once Google treats a URL as a duplicate of a page it has already picked, it comes back to that URL less often, so the recrawl that would show the real content restored is scheduled later than it otherwise would be. The length of the outage does not set the length of the recovery, which is the answer to Holland’s question.
A 200 tells Google that the body it just received is the page. During downtime that is wrong twice over, because the body is not the content and the same wrong body is sitting on every URL on the site. Status codes are the part of a downtime response Google acts on, and its handling of them is finer than up or down: a 4xx on an important URL can keep the URL in search results while the snippet drops. A 503 says the content exists and the server cannot return it right now, which is what is true during an outage.
The thread began with a post from Mueller telling anyone who takes a PHP site down for an unplanned update to answer with 503 rather than let requests time out or return a 404 or 500, and to do that outside PHP, at DNS or the CDN. One reply in the thread gives the practical version of the outside-PHP point: Laravel, a PHP web framework, serves a 503 with a Retry-After header from its php artisan down command, but that runs inside PHP, so a load balancer rule is the safer place for the response when PHP is the thing being replaced.
What to do
- Answer with 503 during short downtime and set Retry-After. Google’s guidance on pausing a site asks for a static informational page on a 503 for a closure of a day or two, with Retry-After carrying a best-effort date or duration.
- Put the maintenance response somewhere the application is not, in a CDN rule, a load balancer rule, or DNS. A maintenance mode that lives in the app cannot answer when the app is what is down.
- Keep robots.txt out of it. The same Google guidance says to carry on allowing crawling in robots.txt and not to serve a 503 for the robots.txt file itself.
- Take the 503 down when you are back. Google’s 2011 post on planned downtime warns that a lasting 503 can eventually be read as the server being permanently unavailable, and URLs can be removed from the index.
- For a closure longer than a couple of days, 200 is the right code, but on a real page. Google’s pause guidance recommends an indexable placeholder home page in that case. The guidance does not spell out why, but a placeholder written for one business is text no other site is serving, so there is nothing for it to be folded in with.