“Error 404 – Page not Found”, this is one of the most common web page types we see in our day-to-day internet surfing. Without any doubt, such pages offer bad user-experience, but how about Search engines? Are 404 pages bad for SEO? Do search engine penalizes website with too many 404 status code pages? In this article, I will look at 404 error pages from search engine and user experience perspective, and will recommend some of the best practices that one can follow to improve their 404 error pages.
Most Common reasons for 404 Error pages:
404 error can happen for multiple reasons, and here I will outline some of the most common once:
- You change your permalink or link structure of your website. (Mostly happens when changing design, or changing permalinks)
- You removed a content from your website
- Somebody linked your website with mis-spelled or wrong link.
When a user or search engine bot request to get access to a page on a server, usually web-servers reply with HTTP status code of 404 to tell page doesn’t exist on the server. Usually search engine bots are smart, and they don’t care about 404 error pages a lot. But when it comes to a user, it’s always a bad user experience, as user usually go back and follow another link. Another disadvantage is, you miss out an important link at times from other domains. This is not something that I worry about much, but when a user land on a website by a referral link, and sees a 404 page, it usually makes them go away.
Finding 404 links and fixing it for Users and SEO:
Now, lets figure out what all you can do with 404 error pages, and how to take out most advantage of it. First lets identify 404 pages on your blog, and the best place to start is with Google Webmaster tool. Login to your Webmaster tool dashboard > Crawl errors > Not found >
You can click on any of the link, and see “linked from” to get an idea, where these pages are linked from. This is also handy to find a potential link juice benefit 404 page on your website. If you are using WordPress, you can use redirection plugin to monitor and redirect 404 error pages from your dashboard. (This is what I use at ShoutMeLoud). Now, once you have a list of 404 pages for your domain, here are few solutions to make most out of it:
- 301 Redirect the link to most relevant post/article or category. If there is no relevant article or category, follow the next step.
- If the error is coming due to mis-spelled link from other domain, you can either ask the webmaster to update the link or 301 redirection is sufficient.
- If no relevant article is on your website related to our 404 link, simply let it be 404. Google will automatically de-index such pages in a while.
- You can also manually de-index such pages from web index using Webmaster removal tool. But, if you have thousands of such pages, it will be a task for you. So, I will rather follow point 3 or next point.
- Use a custom 404 page to spice up your page. You can add a search box, show articles relevant to user search query, or show some of the most popular articles from your portal. The whole idea is to give a good user experience.
Another thing to keep in mind is: If you have a portal with thousands of 404 pages, you wouldn’t like search engine bots to waste their limited crawling resources on such pages, so a good idea is to re-direct when ever possible or block bots from accessing it using Robots.txt, and remove it from Google Webmaster tool. This is ideal, when you have removed any directory or category/tags pages from your portal.
- What is Google crawling and indexing
- Useful tips to increase Google crawl rate
- Understand Google Webmaster tool crawl errors and fixes
404 pages are not SEO enemies, but it’s not a good idea to have a long list of 404 pages on your blog. More over, my main concern with 404 pages is bad user-experience, and with the search engine optimisation revolving around offering good user experience, you might not like to miss a point.
- Learn about soft 404 error
- Use Broken link checker plugin to find broken links on your WordPress blog
- Find broken link on a website using minalyzer
Do let me know how you handle 404 pages on your blog or website? Do you set a redirection or show a custom 404 error page?