HTTP 410 Gone Code Stops Search Engines Re-Crawling Dead Urls
FidoSysop Last Updated
Here is how to use HTTP 410 GONE status code to tell search engines those 404 urls they keep re-crawling over and over again have been permanently removed and will never be back.
If your website or blog is quite old, chances are you have removed posts, added and removed categories, tags, and media attachments over the years.
One of my older blogs has been up since 2007 and has had numerous code updates, themes, plugins, etc. And also went from http to https encryption, and was a nightmare of old dead urls that Google and Bing kept re-crawling over and over again.
While as a general rule 404 urls will not hurt your search rank, all this crawling over and over again looking for material that was removed long ago never to return sucks up server resources. It’s best to redirect those 404s to 410 Gone http status code. This signals bots the url they were requesting is permanently gone and to remove it from their re-crawl list.
This plugin will issue a HTTP 410 response to requests for articles that no longer exist on your blog. When you delete a post or page, it records the URL for that page and issues a 410 response when that URL is requested. You can also manually manage the list of obsolete URLs.
The HTTP Specification defines the 410 response header for use when a resource has been permanently removed. It informs robots visiting your site that the requested content has gone, and that they should stop trying to access it.
410 For WordPress MySQL database table entries
For static html websites this process can be accomplished by adding 410 Gone entries to your .htaccess file. An .htaccess file if configured incorrectly will crash your website hard! Be sure to download it before performing any mods. If you muck up and can’t figure out how, just re-upload the saved copy.
Assuming your website is running on an Apache http server, and mod_rewrite is installed, this works the best for me.
This entry will Wildcard 410 status anything in a tilde directory. RedirectMatch 410 /~/(.*)
This entry will wildcard 410 status anything in this directory. RedirectMatch 410 /foo/(.*)
These entries will 410 status an old unused feed url. RedirectMatch 410 /feed/
RedirectMatch 410 /comments/feed/