How to Deal with the 404 Error for Search Engine Optimization

51Shares
404 Error page SEO
  • Save

“Error 404 – Page Not Found” – This is one of the most common web errors we see in our day-to-day Internet surfing. We know that error pages offer an obviously negative user experience, but how do they impact search engines?

  • Are 404 pages bad for SEO?
  • Do search engines penalize websites with too many 404 status code pages?

In this article, I will look at 404 error pages from the perspective of both users and search engines, and will recommend some of the best practices that one can follow to improve their 404 error pages.

Subscribe on YouTube

Most common reasons for 404 error pages:

404 errors can happen for multiple reasons. Some of the most common are as follows:

  • You change your permalink or the link structure of your website. (This mostly happens when changing design, or changing permalinks.)
  • You removed content from your website.
  • Somebody linked your website to a misspelled link or to an otherwise incorrect link.

When a user or search engine bot requests access to a bad page on a server, web servers usually reply with an HTTP status code of 404, to tell the user that the page doesn’t exist on the server.

Usually search engine bots are smart, and they don’t care much about 404 error pages.  For the real-life user, however,  aa 404 error page presents a bad user experience, and in response the user usually goes back and follows another link.

  • Save

Another disadvantage is of the 404 error is that it can cause you to miss out on important links from other domains. This is not something that I worry much about, but when a user lands on a website by way of a referral link and sees a 404 page, it usually makes them go away.

Let’s figure out what you can do with 404 error pages, and how to deal with them to your best advantage.

We first need to identify any 404 pages on your blog.  The best place to start is with the Google search console (Also known as Google search console). Login to your Webmaster tool dashboard > Crawl errors > Not found >

  • Save

Click on any of the links and you will see “linked from” which will give you an idea of where these pages are linked from. This is also handy for finding potential link juice on your website.

If you are using WordPress, you can use a redirection plugin to monitor and redirect 404 error pages from your dashboard. (This is what I use here at ShoutMeLoud). Once you have a list of 404 pages for your domain, here are several solutions for dealing with them:

  • 301 Redirect the link to the most relevant post/article or category. If there is no relevant article or category, follow the next step.
  • If the error is arising due to a misspelled link from another domain, you can either ask the webmaster to update the link, or you can use the 301 redirection.
  • If no relevant article is on your website related to the 404 link, simply let it be. Google will automatically de-index such pages eventually.
  • You can also manually de-index such pages from the web index using the webmaster removal tool. However, if you have thousands of such pages, it will be quite a task for you. So I would suggest either the previous suggestion or the following one.
  • Use a custom 404 page to spice up your page. You can add a search box, show articles relevant to the user’s search query, or show some of the most popular articles from your portal. The whole idea is to give a good user experience.

Another thing to keep in mind is that if you have a portal with thousands of 404 pages, you do not want the search engine bots to waste their limited crawling resources on such pages.  

For this reason, it is best to re-direct whenever possible or to block bots from accessing these pages using Robots.txt, and remove them using the Google Webmaster tool.

This is ideal particularly when you have removed any directory or category/tag pages from your portal.

404 pages are not SEO enemies, but it’s not a good idea to have a long list of 404 pages on your blog. My main concern with 404 pages is the poor user-experience they create.  With search engine optimization revolving around offering a good user experience, this is an issue we do want to make the effort to address.

Related reading:

How do you handle 404 pages on your blog or website? Do you set a redirection or show a custom 404 error page?  Let us know using the comments section below.

If you find the information in this post useful, please share it with your friends and colleagues on Facebook, Twitter and LinkedIn.

Was this helpful?

Thanks for your feedback!
  • Save
51Shares
Authored By
A Blogger, Author and a speaker! Harsh Agrawal is recognized as a leader in digital marketing and FinTech space. Fountainhead of ShoutMeLoud, and a Speaker at ASW, Hero Mindmine, Inorbit, IBM, India blockchain summit. Also, an award-winning blogger.

34 thoughts on “How to Deal with the 404 Error for Search Engine Optimization”

  1. Shweta

    Hi Harsh, I deleted some of my blog pages, it is still showing in google searches with 404 error. n m unable to fix it. plz share any video if u have.. Thanks

  2. Steve Smith

    Hello Harsh,

    My website was recently hacked. Although I recovered from it but there were around 5000 pages added in my website in the backend. I just got a message from Google Webmasters that there are more than 5000 pages with 404 error on my website. And those 5000 contents are not related to my website’s niche or any of the categories.

    So my question to you is whether to keep those 404 as it is or shall I mark them as fixed? Another point to note is Google says that if you are not aware of any such pages, let them be as it is, it will be de-indexed from the search results. So please let me know what to do in such case. Shall I remove the 404 pages OR let them be as it is OR shall I redirect them to homepage (even if the content is not relevant)?

    And what would be the effect if I keep them as it is. Please note that my website has just 8 posts on it and 5000 404 pages are too much.

    Awaiting for your response Harsh Agarwal

    Regards,
    Steve Smith

  3. deepa

    Thanks for sharing i have delete some post and search engine still showing search results then error shown 404 page no found. I hope i will fix it after reading this article.

  4. Bryan Ripley

    Hi Harsh.

    If I am having many 404 urls and if I mark them as fixed the option which shows in google webmasters so is it fine to do this.

  5. Rafique Mohammad

    Hey Harsh!, thanks for sharing the information about 404 errors. If anybody committed to delete his/her niche less blog, then how can they use this option. Is there any simple way to remove all the links from indexer? I will waiting for your precious reply… thanks again.

  6. Jitendra

    Yes Harsh Jee,
    You are right. I am having problems with 404 errors. I want to redirect them all at once. Redirection plugin is good but it will more better if there is solution that will redirect all the 404 errors page to one single page at once. We should do one by one. Is there any plugin that redirects all the 404 errors page to one single page at once. I am waiting for your happy answer.

  7. Suresh Thakur

    I think to have custom 404 page is the best idea as we can show our articles related to queries as well as it is helpful to make users engaged. Great post with detailed explanation. Your stuff speaks itself.

  8. Gaurav Arora

    Sir I was getting around 450 404 errors and all were coming from China Google Search. So in order to get rid of them I used Smart 404 plugin which simply move all 404 errors to Homepage. Is it fine or I have done something wrong?

  9. bilal

    i really like your solution for 404 pages. but i didn’t understand about the custom 404 error. Do you have an article on it?

  10. Muhammad Imran

    Well, a great article on 404 pages, Thanks Harsh for this informative article.

  11. Gulshan Kumar

    Hi Sir, Is it good idea to setup all 404 to homepage ?

    1. Harsh Agrawal

      @Gulshan
      If it’s too many 404 error pages, I don’t see that as a good practice. If I would have been at your place, I would redirect only those 404 pages which have any link value. You should check out my post on domain authority, it will give you a good insight on Page authority: https://www.shoutmeloud.com/domain-authority.html

  12. Alex Khan

    Hi Harsh,

    I was wondering if you already have a 404 error page setup and if those errors still come up in WMT is it still a bad effect on SEO?

  13. Abhishek Sood

    Hello Harsh,

    Thanks for such an helpful post sir. I was looking for something like this from a long as most of my pages gets deleted from my blog due to which I got so many 404s which ruined the authority. Now your post will help me some to get back/increase the authority of my blog.

    Thanks

  14. Puneet Kaur

    Hi Harsh,
    Your post is very knowledgeable. Almost everyday I face the problem of error 404 on many sites and many times when I surf the internet I get irritated. Your suggestions are very useful and effective to handle such errors. The tool that is suggested above to find broken links is useful for SEO. Thank you for sharing the article.

  15. PolarStar

    Hi Harsh, thanks this is a good and relevant topic. I´ve been dealing with 404s for a while now. I initially had a lot of them because of a Flash banner that google could not recognize for technical reasons, but we solved that.

    Still we have around 500 not found links, this is for a big website, and many of them actually work when you click on the from Google Webmaster Tools. How do you deal with that? From what I´ve been reading there are also server configuration issues that could temporarily create a 404 message for a page that is actually working well.

    Thanks, PS

  16. Mahesh

    Nice piece of information. I want to know if resubmitting sitemap will help to de-index them faster? Is there any plugin to manage the 404 errors, so it may automatically de-index instead of redirecting?

  17. Abhinav Jain

    Hi Harsh,

    i would like to add one more point which i have faced. One of my website i am doing SEO for suddenly started showing up increase in 404 pages. It was like it was showing around 200 pages as 404 error pages in google webmaster and then suddenly it went up to 76ooo , 404 pages in google webmaster. Now first issue was how the hell it can happen when the website is having only 1500 pages and it is showing up 76000 pages as 404 error pages found.

    Well yes the website is actually dynamic website. So chances are some id base pages were getting created from some where. Well i found it as to from where the leakage was happening but there was another issue which was that all those 4o4 pages generated by mistake were showing response status as 200, which means all those pages were working and existing in the eye of google. BIG BIG mistake when we open any of those pages in browser it use to show 404 error message but actually those pages were not showing server response status as 404.

    Lesson learned even if your error page is showing 404 message you also need to check it page response status as 404 if it is not showing you need to set it.

  18. Pramod

    I had number of 404 errors and I redirected all of them to related pages before applying for google adsense and after few days I got mail from adsense team with rejection of my application and the one of the reason was somewhat like “You have number of redirects”.
    After that I searched lot about impact of 404 errors pages on seo but I found that they have no such impact but then why google adsense rejected my aplication?Do we need to completely remove all such pages before applying to adsense?

  19. stargaterich

    The redirect plug-in is the easiest and fastest method to resolve the 404 error pages. I think one way to monetize a blog is to redirect it to an affiliate sales page. Thanks for the great content in this post.

  20. Heru Prasetyono

    I think thie error message (404-error page not found) has closely related to the worst service of the web host. It often happened to my blog on my initial start blogging.
    I experienced this for almost one month and this drove me mad and frunstrated.
    Until I finally decided to migrate to a new web host service which I can rely on and it serves me well up to now. My first web host service was really disappoited me.
    And I wrote this bad experience in one of my blog posts.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top
51 Shares