This is a part of our Question and Answer series and if you have any question related to Blogging, SEO and WordPress, feel free to drop your queries via contact form. Selected question will be answered via Blog post. You can also check out previous questions answered here at ShoutMeLoud.
Today’s Question: This is a question by Irfan Siddiqui from realtimetricks.com and he is facing issues with Site indexing. He owns a WordPress blog and despite of having noindex tag on his Tags and categories pages, Google has indexed WordPress tags and categories pages.
Google Indexing Issue with WordPress Attachment, tags & categories ?
This is one of the common indexing issue where Google indexes few pages from your blog and despite of adding noindex tag, such pages are part of Google. This is normal and as soon as Google will recrawl your site, it will respect the noindex tag and will deindex such part of your Blog. This may take some time but here are few things which you can do to speed up the process.
Make sure your Robots.txt is not blocking access to tags and categories pages, as best way to stop indexing is using Noindex tag. If few of your pages are already indexed in Google and you have added noindex tag and also stopped crawling of that part of your site, Google may not recrawl particular section of your site and will never see noindex tag. Robots.txt exclusion is useful for parts like Wp-admin and Wp-content.
- Useful tips to increase Google crawl rate
- Use Robots.txt to protect your blog from Duplicate content issue
More over, it’s not a good idea to stop crawling of pages like tags and categories using robots.txt as it will hamper the crawling of your deeper pages. The best way to stop indexing is by adding noindex tag, which you can easily do by using WordPress SEO by yoast or meta robots plugin.
Now in the case of Irfan blog, he has not blocked the tags and categories pages using Robots.txt but before adding noindex tags, Google have already indexed those pages. Here is a quick solution for anyone facing the similar problem.
You can check what pages of your sites are indexed in Google by typing site:domain.com in Google search.
Make sure you use the noindex option for tags and categories pages in WordPress SEO by yoast or Meta robots plugin. If you are using Thesis theme, you can find this option under Site option > Document head > Robots meta tag > Noindex categories and tags.
- Login to Google Webmaster tool
- Site dashboard > Optimization > Remove URL’s
Now usually our tags and categories pages are seen as directory from search point of view. For example:
- domain.com/tag/SEO
- doamin.com/tag/WordPress
We can use the directory removal option to remove the complete directory from noindex tag and within 24-48 hours all your categories and tag pages will be out of Google index.
So you need to add www.domain.com/tag/ and www.domain.com/category/ and select Remove directory option , hit submit request and you got rid of tags and categories pages from search index.
Related reading:
- What is common WordPress duplicate content issue and fixes
- How to noindex WordPress categories and tags
Getting rid of WordPress attachment ID links
This is another thing which not only create duplicate content issue but also add low quality content in Google search. We can again fix this issue by using Meta robots or WordPress SEO by yoast plugin. Though, here is a practice which you should always try to maintain. When you upload an image, make sure your Image link URL is either blank or if you want to make it clickable, use link to image option. See below screenshot for detailed explanation:
This will ensure no such links will be indexed in future and additionally here are configuration settings for plugins which I mentioned above:
Robots meta plugin settings:
Settings > Robots meta settings > Permalink Settings > Put a tick mark on “Redirect attachment URL’s to parent post URL.”
WordPress SEO by yoast setting:
WordPress dashboard > SEO > Permalink Settings > Put a tick on “Redirect attachment URL’s to parent post URL.”
That’s it and it will redirect all such >attachment_id= links to respective post and in few days Google will deindex all such links.
I hope this quick solution will help you anybody facing the similar issue. Do let me know, if you ever faced the similar WordPress indexing issue like irfan?
I am in a bit of confusion if i should disallow category or not,i see some errors in html improvements area with duplicate tittle tag with categories ….disallowed tag, archives,page etc but im not sure with category …
Srikar
For a normal blog, I would recommend it to keep it deindex as it will create a duplicate content issue.
if it’s a micro-niche site, it all depends upon how you planning your site SEO. Usually in micro-niche sites, we have less content and static content where we can also take advantage of categories for SEO, but for a normal blog it’s not recommended as it not only create low quality content issue but also create duplicate content issue. As you already mentioned you are seeing duplicate title tag issue here.
yeah,i have been hit by panda couple of times and lately it makes me wonder if its the trailing slash thing and this categories,tags and pagination pages that is causing the duplicate content issue,i do not see any other reason coz with content i have tried my best and have moderated to follow the quality guidelines,so it just makes sense that this is the root of all issue
Thank you so much for the article on my question I didn’t expected that I’ll get a nice tutorial on my question.
I just have removed that tag from being indexed into Google using webmaster tool all credit goes to your tutorial. I hope within 48 hours it will be deindex.
Recenlty i also faced this kind of issue. If everyone faced this problem, then its coz of Algorithms for sure. I spoke with Srikar few days back about this and hope we able to solve this to some extent. I stumbled upon this post on seomoz http://www.seomoz.org/blog/setup-wordpress-for-seo-success
The best post i can say, Even though we are very good at WP but checking the post which makes fundamentals even more stronger. I recommend everyone to check it .
i am suggesting all my friends to not deindex it.
because it is may drive some good organic traffic to your blog.
but many times it disguide visitors, at that time your indexed tags sucks.
so, it depend on your blog’s category….
No indexing tags, categories and archives is nowadays most important part of SEO. Also it is important nowadays to repair old crap content or completely get it de-indexed it using noindex tag.
Nice article on a common problem. Many people are facing this problem. Thank you for guidance and sharing a nice tutorial.
This is really useful and you made it more clear by describing how to configure attachment ID links using Robots meta and Yoast plugins. Thanks Harsh for this wonderful and useful post
My blog categories are already deindexed from google with help of Yoast SEO plugin.
Thanks for sharing
Yes, we should deindex our tags, categories and attachments from the search engines so that the main post can get more traffic and accordingly the users can get the post they have been looking for. I usually use All In One SEO, and these things can easily be deindexed from the search engines using this WordPress plugin.
Thank you for this detailed post on deindexing the categories and tags. almost all my tags and categories are indexed i will use meta robots plugin for deindexing.
Great sharing Bro.. My very first site also got categories and tags indexed in Google, for which I was thinking to find a solution.
Your tips will help me to de-index those.
Thank you for this information. It really helps me regarding how to deindex my tags and categories in google search, and I really thought it was just fine when they show up in search engines though I really don’t like them to show up, but I really don’t know how to deindex them not until now. I have already WP SEO by Yoast installed and I just follow what you posted. Thanks.
Thanks Harsh,
Trying to deindex my tags and categories from Google Search, using Webmasters tool.
I had a question : I have recently changed the Meta, description of my blog quite a few times but can’t see the same updated in Google search. Is there any frequency factor for that?
Great info,, But in my case, i have a photoblog containing alot of images and all the attachment pages are indexed by google(more than 10k) and recently i am affected by Panda update.
Now, obviously i don’t want to redirect my attachment page to parent post because its a photo blog and also i want to deindex those attachment pages and don’t wanna allow search engines to index my further uploaded images.
and also using noindex tag will my images be removed from Google Images?
Hi Hars…
I have installed Robots meta plugin, but there is no “Robots meta settings” in settings menu, I use wordpress 3.5.1 versions. How can I solve it, please help me..
Thanks….
@Achmad
It should be under settings > Robots meta
If you are using WordPress SEO by yoast plugin, that plugin also have all the options which you need here.
I have one question, i’m using google xml sitemap generator in my blog, can i use Yost plugin at a time, to prevent indexing of these categories, tags, attachment images etc…
will it cause issue?. please help me.
Thanks
@MAdhu
No it will not. Using Yoast you can noindex categories, tags, author archive and it will not impact your sitemap file in any ways.
I have enabled “Redirect attachment URL’s to parent post URL.” in Yoast but I do not see any difference. All the attachments being indexed is starting to annoy me and I don’t fancy disallowing them one by one in robots.txt.
Have I gone wrong somewhere?
@It will take some to get those attachment links unindexed… Just Make sure those attachment links are redirecting to parent post link…