If you know how Search engine bots work, crawling and indexing will sound familiar to you. If not, I will explain this topic in-depth here. First of all Google does not provide any guarantees or promises that they will crawl or index your blog stuff. You must keep maintaining your blogs content to enter into Google’s boundaries. Google has a certain amount of resources to crawl, Index and Rank your sites. And it has to treat them differently according to the popularity and importance of the site. So let’s have a brief look at How Google does all these things.
Understand Search engine bots Crawling and Indexing:
What is Crawling
Google spiders will crawl your site when you post some thing in your site. it may not happen at the instant because it is based on a lot of factors. Google does not have any human workers for crawling sites, They are using a computer algorithm for this purpose. So it must consider the factors such as Page Rank, Back Links etc before crawling. Another thing you can do for faster crawling is with sitemaps. But Google will not crawl anything you put in your sitemap. Sitemap will just let the Google know about your site contents. Bonus tip: Put your sitemap link at the footer of your blog. This will ensure that Google bot will follow your sitemap.
What is Indexing
After your content get crawled it ‘may’ be indexed. Before indexing Google want to make sure that your content is worth it. Sometimes they are not, That may be because of several reasons.
- Internal Duplication
- External Duplication
- Weak Contents
So make sure your content is unique ie not copied from any existing contents.
How long does Google take for indexing ?
It may varies, It depends on how your content interlinked, How often you update your site content. For new sites, It will take weeks or even months for indexing.
How to get indexed faster in Google Search ?
For new blogs, It may take some time. So take a long breath and be patient. Google will rate your blog according to your updating frequency, backlinks etc. For older blogs, If you are still facing indexing problem, make sure you do not have internal or external duplication of contents. And do not rely on sitemaps. Because they are optional. Make sure the contents can be searched via normal links.
Many of the newbie bloggers are getting struggled with the Google’s indexing concept. Have a look at this table(Not accurate just for a reference)
Do let us know how long dies it take for Google bots to crawl and index your blog? Have you been facing indexing issue lately?
This is a guest post by Jimmy who blogs at zxeen. If you would like to write for ShoutMeLoud, check our guest posting guidelines.
Subscribe Updates, Its FREE!
→ 
→ 
{ 26 comments… read them below or add one }
Interesting chart you put.
Once your blog established well, it can be indexed within minute, I experienced it. Its mostly connected to uniqueness of content.
I think you missed an important factor, which is Robots.txt file. If this file is blocking Google bots, then no pint in trying other all factors.
Thanks for the nice post Jimmy
Yup, you are right Dinesh
Thanks
Awesome post dude.I was confused and worried that my blog is not indexed by Google.I will be now patient .
My site is also not getting crawled faster so i have to update my blog frequently. this is really useful.
When I post a new post, Google index it after 1 minute
Writing post usually is the good way to get indexed faster
Thanks for the detail mate! That analysis looks great =D
Getting indexed on google is a lot easier than it is on some other search engine like Bing. Another thing is that if you make a blog on blogger.com you are almost always likely to get indexed by google.
i think i am too a victim of slow indexing by Google…
need to curb it fast
I it happened with my earlier site…but i got the problem fixed
Great post buddy, indexing is the problem for most of the bloggers, thanks for clarifying the concepts…
Well my blog is Google friendly.With in minutes my posts get indexed in Google search engine…I get 70% of traffic from google
Google sometimes index your new blog post intent also probably most of my blog from few months got intent indexed…
I have a new blog too..Thanks for the great info. came hand, when I was pondering why my pages are not getting indexed.
That table is very useful.
I too have to update my blog frequently and on regular basis. It will satisfy my readers and Google as well.
As it is always said: ”If content is the King, then SEO is Queen”
indexing in google is very tough task…but my blog indexed in google within a week after i submitted sitemap to google webmasters tools….
I think if people focus more on creating good content verses indexing, they will soon realize it won’t be long until the site grows.
That doesn’t mean disregard indexing, but rather focus on content first, then SEO.
Hey Jimmy,
Thanks for that unique Chart. Really helpful post!
Great post! This will help lot of new bloggers to start having patience.
Even for popular blogs there are some posts which don’t get indexed. It is due to the fact that they might be copied from somewhere or doesn’t add too much value. This could be the reason.
your table is good. Can i know how did you come up with that table?
Hi Nihar,
Thanks for your reply
I was experienced this problem earlier and i got reply from a google webmaster forum moderator with this table.
thank you for sharing it..
try find why my new domain (using blogspot) slow index by google not like old domain.
now i know why
Its going to be a month,Google has not indexed my website… any suggestions
help required…
I am also facing from slow indexing issue. Can you review my site and give me feedback to improve my indexing status.
From November , My post are not crawl (googleboot) and in Webmaster Tools a message i.e,
Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn’t crawl any pages listed in that file, we postponed our crawl. Your site’s overall robots.txt error rate is 100.0%.
You can see more details about these errors in Webmaster Tools.
Recommended action
If the site error rate is 100%:
Using a web browser, attempt to access http://www.speedorock.com/robots.txt. If you are able to access it from your browser, then your site may be configured to deny access to googlebot. Check the configuration of your firewall and site to ensure that you are not denying access to googlebot.
If your robots.txt is a static page, verify that your web service has proper permissions to access the file.
If your robots.txt is dynamically generated, verify that the scripts that generate the robots.txt are properly configured and have permission to run. Check the logs for your website to see if your scripts are failing, and if so attempt to diagnose the cause of the failure.
If the site error rate is less than 100%:
Using Webmaster Tools, find a day with a high error rate and examine the logs for your web server for that day. Look for errors accessing robots.txt in the logs for that day and fix the causes of those errors.
The most likely explanation is that your site is overloaded. Contact your hosting provider and discuss reconfiguring your web server or adding more resources to your website.
If your site redirects to another hostname, another possible explanation is that a URL on your site is redirecting to a hostname whose serving of its robots.txt file is exhibiting one or more of these issues.
After you think you’ve fixed the problem, use Fetch as Google to fetch http://www.speedorock.com/robots.txt to verify that Googlebot can properly access your site.
Learn more in our Help Center.
pls reply as soon as possible.
Hi, jimmy Excellent post. google indexing my site privacy policy, contact us and dcma policy page. is it creates problem or not?
@Raj
It won’t create any problem but such pages are not useful for search engines, but is useful for user experience. You should consider adding noindex tag for such pages…
Google indexing blog post time is really depending on many things like blog age, sitemap, page rank and backlinks. But aside from these elements I suggest that you focus on your content and Google indexing will no longer be a problem.