ShoutMeLoud – Shouters Who Inspire

Superlinks
≡ Menu
≡ Menu

404 Error : How to Deal for Search Engine Optimization?

bluehost
404 Error : How to Deal for Search Engine Optimization?

“Error 404 – Page not Found”, this is one of the most common web page types we see in our day-to-day internet surfing. Without any doubt, such pages offer bad user-experience, but how about Search engines? Are 404 pages bad for SEO? Do search engine penalizes website with too many 404 status code pages? In this article, I will look at 404 error pages from search engine and user experience perspective, and will recommend some of the best practices that one can follow to improve their 404 error pages.

404 Page not found

Most Common reasons for 404 Error pages:

404 error can happen for multiple reasons, and here I will outline some of the most common once:

  • You change your permalink or link structure of your website. (Mostly happens when changing design, or changing permalinks)
  • You removed a content from your website
  • Somebody linked your website with mis-spelled or wrong link.

When a user or search engine bot request to get access to a page on a server, usually web-servers reply with HTTP status code of 404 to tell page doesn’t exist on the server. Usually search engine bots are smart, and they don’t care about 404 error pages a lot.  But when it comes to a user, it’s always a bad user experience, as user usually go back and follow another link. Another disadvantage is, you miss out an important link at times from other domains. This is not something that I worry about much, but when a user land on a website by a referral link, and sees a 404 page, it usually makes them go away.

Finding 404 links and fixing it for Users and SEO:

Now, lets figure out what all you can do with 404 error pages, and how to take out most advantage of it. First lets identify 404 pages on your blog, and the best place to start is with Google Webmaster tool. Login to your Webmaster tool dashboard > Crawl errors > Not found >

find 404 error pages

You can click on any of the link, and see “linked from” to get an idea, where these pages are linked from. This is also handy to find a potential link  juice benefit 404 page on your website.  If you are using WordPress, you can use redirection plugin to monitor and redirect 404 error pages from your dashboard. (This is what I use at ShoutMeLoud). Now, once you have a list of 404 pages for your domain, here are few solutions to make most out of it:

  • 301 Redirect the link to most relevant post/article or category. If there is no relevant article or category, follow the next step.
  • If the error is coming due to mis-spelled link from other domain, you can either ask the webmaster to update the link or 301 redirection is sufficient.
  • If no relevant article is on your website related to our 404 link, simply let it be 404. Google will automatically de-index such pages in a while.
  • You can also manually de-index such pages from web index using Webmaster removal tool. But, if you have thousands of such pages, it will be a task for you. So, I will rather follow point 3 or next point.
  • Use a custom 404 page to spice up your page. You can add a search box, show articles relevant to user search query, or show some of the most popular articles from your portal. The whole idea is to give a good user experience.

Another thing to keep in mind is: If you have a portal with thousands of 404 pages, you wouldn’t like search engine bots to waste their limited crawling resources on such pages, so a good idea is to re-direct when ever possible or block bots from accessing it using Robots.txt, and remove it from Google Webmaster tool. This is ideal, when you have removed any directory or category/tags pages from your portal.

404 pages are not SEO enemies, but it’s not a good idea to have a long list of 404 pages on your blog. More over, my main concern with 404 pages is bad user-experience, and with the search engine optimisation revolving around offering good user experience, you might not like to miss a point.

Related read:

Do let me know how you handle 404 pages on your blog or website? Do you set a redirection or show a custom 404 error page?

{ 6 comments… add one }

  • PolarStar

    Hi Harsh, thanks this is a good and relevant topic. I´ve been dealing with 404s for a while now. I initially had a lot of them because of a Flash banner that google could not recognize for technical reasons, but we solved that.

    Still we have around 500 not found links, this is for a big website, and many of them actually work when you click on the from Google Webmaster Tools. How do you deal with that? From what I´ve been reading there are also server configuration issues that could temporarily create a 404 message for a page that is actually working well.

    Thanks, PS

    Reply
  • Mahesh

    Nice piece of information. I want to know if resubmitting sitemap will help to de-index them faster? Is there any plugin to manage the 404 errors, so it may automatically de-index instead of redirecting?

    Reply
  • Abhinav Jain

    Hi Harsh,

    i would like to add one more point which i have faced. One of my website i am doing SEO for suddenly started showing up increase in 404 pages. It was like it was showing around 200 pages as 404 error pages in google webmaster and then suddenly it went up to 76ooo , 404 pages in google webmaster. Now first issue was how the hell it can happen when the website is having only 1500 pages and it is showing up 76000 pages as 404 error pages found.

    Well yes the website is actually dynamic website. So chances are some id base pages were getting created from some where. Well i found it as to from where the leakage was happening but there was another issue which was that all those 4o4 pages generated by mistake were showing response status as 200, which means all those pages were working and existing in the eye of google. BIG BIG mistake when we open any of those pages in browser it use to show 404 error message but actually those pages were not showing server response status as 404.

    Lesson learned even if your error page is showing 404 message you also need to check it page response status as 404 if it is not showing you need to set it.

    Reply
  • Pramod

    I had number of 404 errors and I redirected all of them to related pages before applying for google adsense and after few days I got mail from adsense team with rejection of my application and the one of the reason was somewhat like “You have number of redirects”.
    After that I searched lot about impact of 404 errors pages on seo but I found that they have no such impact but then why google adsense rejected my aplication?Do we need to completely remove all such pages before applying to adsense?

    Reply
  • stargaterich

    The redirect plug-in is the easiest and fastest method to resolve the 404 error pages. I think one way to monetize a blog is to redirect it to an affiliate sales page. Thanks for the great content in this post.

    Reply
  • Heru Prasetyono

    I think thie error message (404-error page not found) has closely related to the worst service of the web host. It often happened to my blog on my initial start blogging.
    I experienced this for almost one month and this drove me mad and frunstrated.
    Until I finally decided to migrate to a new web host service which I can rely on and it serves me well up to now. My first web host service was really disappoited me.
    And I wrote this bad experience in one of my blog posts.

    Reply

Leave a Comment