This is a part of our Question and Answer series. If you have any questions related to Blogging, SEO, and/or WordPress, feel free to drop your queries via our contact form. Selected questions will be answered via a blog post. You can also check out previous questions answered here at ShoutMeLoud:
- A Few Blogging Questions & My Answers
- How To Become A Professional Blogger
This is a question by Irfan Siddiqui from realtimetricks.com.
Today’s Question:
“I am facing issues with site indexing. I own a WordPress blog and despite having ‘noindex’ tags on all of my tags and categories pages, Google has indexed WordPress tags and categories pages anyway. How can I deindex these tags and category pages?”
Google Indexing Issue with WordPress Attachments, Tags & Categories
This is one of the common indexing issues where Google indexes a few pages from your blog despite you adding a noindex tag. This is normal, and as soon as Google recrawls your site, it will respect the noindex tag and will deindex these parts of your blog.
This may take some time, but here are a few things which you can do to speed up the process.
First, make sure your Robots.txt is not blocking access to the tags and categories pages.
If a few of your pages are already indexed in Google and you have added the noindex tag, Google may have stopped crawling of that part of your site and never see the noindex tag because Robots.txt is blocking access to crawlers.
Robots.txt exclusion is only useful for parts like Wp-admin and WP-content.
- Useful Tips To Increase Google Crawl Rate
- Use Robots.txt To Protect Your Blog From A Duplicate Content Issue
Moreover, it’s not a good idea to stop crawling pages like tags and categories using robots.txt as it will hamper the crawling of your deeper pages. The best way to stop indexing is by adding a noindex tag, which you can easily do by using WordPress SEO by Yoast.
Now in the case of Irfan’s blog, he has not blocked the tags and categories pages using Robots.txt, but before adding noindex tags, Google has already indexed those pages.
Here is a quick solution for anyone facing a similar problem.
First, you can check what pages of your site is indexed in Google by typing this search operator into Google:
- site:domain.com (replace domain.com with your domain)
Make sure you use the noindex option for tags and categories pages in WordPress SEO by Yoast or the Meta Robots plugin.
If you are using the Thesis theme, you can find this option under Site option > Document head > Robots meta tag > Noindex categories and tags.
- Login to your Google Webmaster Tool
- Go to Site Dashboard > Google Index > Remove URL’s
Now, usually our tags and categories pages are seen as a directory from a search engine’s point of view.
For example:
- domain.com/tag/SEO
- doamin.com/tag/WordPress
We can use the directory removal option to remove the complete directory from the noindex tag. Within 24-48 hours, all of your categories and tag pages will be out of Google’s index.
So you need to add www.domain.com/tag/ and www.domain.com/category/ and select “Temporarily hide directory”.
Hit “Submit Request” and now you’ve gotten rid of tags and categories pages from Google’s search index.
Related reading:
Getting rid of WordPress attachment ID links
This is another thing which not only creates duplicate content issues, but also adds low-quality content into Google Search. We can again fix this issue by using the Yoast SEO WordPress plugin.
Also, here is a practice that you should be following: When you upload an image, make sure your image link URL is either blank, or if you want to make it clickable, use the “Link to Image” option.
See this screenshot:
This will ensure no such links will be indexed in the future.
Here are configuration settings for plugins which I mentioned above:
WordPress SEO by Yoast setting:
WordPress SEO dashboard > Advanced > Permalinks > Put a tick on “Redirect attachment URLs to parent post URL.”
That’s it and it will redirect all such >attachment_id= links to its respective post. In a few days, Google will deindex all such links.
I hope this quick solution will help anybody facing a similar issue. Do let me know if you’ve ever faced a similar WordPress indexing issue like Irfan. Did this solution fix the problem? Let me know in the comments!
Thanks. With it I deindexed category and tags from search. One more question, is it better to deindex author pages from search.
is this the same way to removetag and catagories for blogger blog?
I have enabled “Redirect attachment URL’s to parent post URL.” in Yoast but I do not see any difference. All the attachments being indexed is starting to annoy me and I don’t fancy disallowing them one by one in robots.txt.
Have I gone wrong somewhere?
@It will take some to get those attachment links unindexed… Just Make sure those attachment links are redirecting to parent post link…
I have one question, i’m using google xml sitemap generator in my blog, can i use Yost plugin at a time, to prevent indexing of these categories, tags, attachment images etc…
will it cause issue?. please help me.
Thanks
@MAdhu
No it will not. Using Yoast you can noindex categories, tags, author archive and it will not impact your sitemap file in any ways.
Hi Hars…
I have installed Robots meta plugin, but there is no “Robots meta settings” in settings menu, I use wordpress 3.5.1 versions. How can I solve it, please help me..
Thanks….
@Achmad
It should be under settings > Robots meta
If you are using WordPress SEO by yoast plugin, that plugin also have all the options which you need here.
Great info,, But in my case, i have a photoblog containing alot of images and all the attachment pages are indexed by google(more than 10k) and recently i am affected by Panda update.
Now, obviously i don’t want to redirect my attachment page to parent post because its a photo blog and also i want to deindex those attachment pages and don’t wanna allow search engines to index my further uploaded images.
and also using noindex tag will my images be removed from Google Images?
Thanks Harsh,
Trying to deindex my tags and categories from Google Search, using Webmasters tool.
I had a question : I have recently changed the Meta, description of my blog quite a few times but can’t see the same updated in Google search. Is there any frequency factor for that?
Thank you for this information. It really helps me regarding how to deindex my tags and categories in google search, and I really thought it was just fine when they show up in search engines though I really don’t like them to show up, but I really don’t know how to deindex them not until now. I have already WP SEO by Yoast installed and I just follow what you posted. Thanks.
Great sharing Bro.. My very first site also got categories and tags indexed in Google, for which I was thinking to find a solution.
Your tips will help me to de-index those.
Thank you for this detailed post on deindexing the categories and tags. almost all my tags and categories are indexed i will use meta robots plugin for deindexing.
Yes, we should deindex our tags, categories and attachments from the search engines so that the main post can get more traffic and accordingly the users can get the post they have been looking for. I usually use All In One SEO, and these things can easily be deindexed from the search engines using this WordPress plugin.
My blog categories are already deindexed from google with help of Yoast SEO plugin.
Thanks for sharing 🙂
This is really useful and you made it more clear by describing how to configure attachment ID links using Robots meta and Yoast plugins. Thanks Harsh for this wonderful and useful post 🙂
Nice article on a common problem. Many people are facing this problem. Thank you for guidance and sharing a nice tutorial. 🙂
No indexing tags, categories and archives is nowadays most important part of SEO. Also it is important nowadays to repair old crap content or completely get it de-indexed it using noindex tag.
i am suggesting all my friends to not deindex it.
because it is may drive some good organic traffic to your blog.
but many times it disguide visitors, at that time your indexed tags sucks.
so, it depend on your blog’s category….
Recenlty i also faced this kind of issue. If everyone faced this problem, then its coz of Algorithms for sure. I spoke with Srikar few days back about this and hope we able to solve this to some extent. I stumbled upon this post on seomoz http://www.seomoz.org/blog/setup-wordpress-for-seo-success
The best post i can say, Even though we are very good at WP but checking the post which makes fundamentals even more stronger. I recommend everyone to check it .
Thank you so much for the article on my question I didn’t expected that I’ll get a nice tutorial on my question.
I just have removed that tag from being indexed into Google using webmaster tool all credit goes to your tutorial. I hope within 48 hours it will be deindex.
yeah,i have been hit by panda couple of times and lately it makes me wonder if its the trailing slash thing and this categories,tags and pagination pages that is causing the duplicate content issue,i do not see any other reason coz with content i have tried my best and have moderated to follow the quality guidelines,so it just makes sense that this is the root of all issue 😐
I am in a bit of confusion if i should disallow category or not,i see some errors in html improvements area with duplicate tittle tag with categories ….disallowed tag, archives,page etc but im not sure with category …
Srikar
For a normal blog, I would recommend it to keep it deindex as it will create a duplicate content issue.
if it’s a micro-niche site, it all depends upon how you planning your site SEO. Usually in micro-niche sites, we have less content and static content where we can also take advantage of categories for SEO, but for a normal blog it’s not recommended as it not only create low quality content issue but also create duplicate content issue. As you already mentioned you are seeing duplicate title tag issue here.