• X

    (not provided) – The Alternative Path to Organic Keyword Data

    By in SEO

    Bluehost hosting

    As a direct result of Google switching to a secure connection (https) and encrypting user searches, we have been witnessing how an increasing amount of keyword data in Google Analytics fell under the (not provided) category.

    Google Not Provided Keyword (not provided)   The Alternative Path to Organic Keyword Data

    While in November 2011 about 12% of search queries were ‘not provided’, the percentage rose to 50% by August 2013, 80% by December 2013 and is closing in on 100%. (source: http://www.notprovidedcount.com/)

    Slowly, we reached the moment we were so terrified of, that moment when search data no longer made it back to us, creating a huge void in our analysis.

    The tragic moment of 100% (not provided) Keywords

    When (not provided) takes up the majority of your keyword list, you’re faced with a whole new set of challenges. You need to find a workaround for every task in your routine that was relying on organic keyword data.

    The greatest impact this change had on SEO was the fact that it crippled your ability to measure, assess and report the effectiveness of your work to your clients or employers.

    Beside this, without keyword data, you are no longer allowed to segment users by keywords directly within your web analytics software, branded vs. non-branded keyword trends can no longer be monitored. You are now faced with the impossibility to pinpoint new or trending search terms, that could potentially generate traffic for your website.

    Can we find our way back to Not Provided search queries?

    Luckily, these search queries are not completely hidden from us. There are other ways in which you can access organic search data for your websites:

    • Google Webmaster Tools has a significant amount of keyword-specific data available, including keywords from encrypted search queries. By linking GWT with Google Analytics (GA) you can also access this data directly from GA.
    • Google AdWords displays specific keyword data for paid campaigns (PPC), but you can easily rely on this information when setting up your organic campaigns also.
    • Non-Google search engines still pass some keyword data back to webmasters. However, by March 31, 2014 all Yahoo properties will also switch to encrypted search, making search data unavailable. As for Bing, although it has already switched to secure search, it is not the default option for users, therefore only a small amount of searchers use it.
    • Site searches also can give you a pretty good idea about the keywords your visitors are using when searching for your products or services. If you need help setting up site search for your website, this article gives great instructions for this.

    For a more complex view over how different tools can provide you with a piece of the missing information, I recommend you to also check out these great articles:

    Can we regain the ability to report?

    Although as SEO professionals we are used to mixing together the data we gather from multiple sources, these new keyword data sources that we are now forced to use bring something new to the table.

    Thanks to (not provided) search queries, in order to still be able to prove to your clients that your optimization efforts paid off and translated into traffic and conversions, you’ll need to reconfigure your entire reporting flow.

    Let’s break things down:

    Step one – Gather and secure the data

    Being the next best thing compared to what we used to have before the (not provided) era, Google Webmaster Tools will probably become your primary keyword data source now.

    To efficiently use this data, you should first of all know that this information is only available in your GWT account for 90 days, so constant, timely, backup is required.

    For this, GWT offers a direct ‘Download’ option, for both the raw data and the charts, either as CSV file or to your Google Drive. However, it lacks the ability to automate this task by schedule, therefore you can either perform it manually every three months or link your GWT account to an additional monitoring system to pull the information for you.

    Keep in mind though that linking GWT with GA does not secure the data for more than the 90 day standard period.

    Step two – Compiling reports

    Since keyword data served multiple purposes in your reporting flow, there are several types of reports that have been affected by the (not provided) change. We’ll further discuss the three most common types of such reports:

    • The evolution of organic keyword traffic

    The metrics GWT provides for organic keywords are different than those found in GA, changing the way keyword traffic is estimated:

    GWT search queries (not provided)   The Alternative Path to Organic Keyword Data

    With only Impressions, Clicks, CTR and Average Position metrics available for each search term, the Clicks metric remains a sole indicator of keyword performance.

    Although at first sight this might seem similar to the old traffic metric, you should note that the Clicks value is different from the actual Visits value. This has to do with the way Google counts them. For example, it is counted as a click but not a visit in cases like: when an user clicks your website in search results but closes the page before it’s fully loaded (counts as one click and no visits), when an user clicks your website twice in search results in a 30 minutes time frame (counts as two clicks but only one visit) or when users who don’t have JavaScript enabled in their browser visit your site from search results (GA tracks only visits from users with JavaScript enabled).

    • Brand awareness monitoring (branded vs non-branded keywords)

    Segmenting the branded from the non-branded traffic driven to your website has also changed, being restricted now only to the data provided by GWT. Even if you import the data from GWT to GA, you can’t create advanced segments for determining the branded traffic like you used to do before (not provided). Therefore, the only solutions remaining are that you either process the CSV exports manually or use a dedicated software to manage your keyword list.

    Just about any type of spreadsheet should allow you to create advanced filters for segmenting the daily traffic (clicks) into branded and non-branded and get something somewhat similar to this:

    Branded vs unbranded keywords AWR cloud (not provided)   The Alternative Path to Organic Keyword Data

    • Identify poorly optimized terms and pages

    Basically, by comparing the keywords that are actually bringing you traffic with their performance in SERPs you are able to identify profitable, but poorly optimized for, terms to further focus your efforts on. Although GWT’s reports provide you with relevant information for this task, you need to pay close attention to a few aspects.

    First of all, note that the information you see in the Top Queries report is only relevant at website level:

    “Impressions—the number of times any URL from your site appeared in search results viewed by a user, not including paid AdWords search impressions. Clicks—the number of clicks on your website URLs from a Google Search results page, not including clicks on paid AdWords search results. [...] In the case where a query returns more than one URL from your site, the average position is based on the most prominent URL in the search results when only a low number of URLs from your site are displayed.”

    (source: https://support.google.com/analytics/answer/1308626?hl=en)

    What it means is that if you are struggling to promote a landing page on your site for a specific keyword, but your website index page for example also ranks for that term on a higher position, the information provided by GWT in the Top Queries report will not be relevant for evaluating the performance of your landing page. You will see the Average position of your Index page and the Clicks and Impressions of all your website pages ranking for that term.

    If you wish to see this information tailored to each ranking URL, you need to click the desired query and access the detailed report. For a complete view, you need to download all detailed reports and merge them into a separate spreadsheet.

    Secondly, the Average position is not just a general metric, calculated for all URLs of a website, but it is also the average of top positions:

    “To calculate average position, we take into account the top ranking URL from your site for a particular query. For example, if Jane’s query returns your site as the #1 and #2 result, and David’s query returns your site in positions #2 and #7, your average top position would be 1.5.”

    (source: https://support.google.com/webmasters/answer/35252?hl=en)

    Therefore, the average position is only relevant for the utopian case when your website ranks for each keyword only with the landing page you are specifically optimizing and tracking. We all know that this rarely happens, and you will often find the Average position metric to be insufficient in your analyses.

    Ideally you would want to use updated keyword rankings and search volumes added to the data retrieved from Google Website Tools all together in a report, to be able to identify, for example, a lower-ranking keyword that however amasses a large number of monthly searches but only a few actual clicks.

    Organic keyword data report AWR cloud (not provided)   The Alternative Path to Organic Keyword Data

    Conclusion

    Although (not provided) seemed to leave us in the dark, in the end things are not that bad. Google Webmaster Tools is quite a formidable tool, that leveraged properly and paired with additional sources of data, can provide you with relevant information to answer all your needs. For further reading:

    But what about you? Have you developed any workarounds for the (not provided) apocalypse? I look forward to your stories in the comments section below.

    This masterpiece is by Dana Loiz. If you would like to submit an original article, check our submission guidelines.

    Subscribe to Download WordPress Guide Worth $99

    Discover more awesome articles

    Article by

    Dana has written 2 articles.

    If you like This post, you can follow ShoutMeLoud on Twitter. Subscribe to Blogging tips via RSS or EMAIL to receive instant updates.

    { 12 comments… read them below or add one }

    Pramod

    Awesome post, but I still didn’t understand why is Google not providing this day to us? Is there any chances for spamming activity? About gwt, the data it provides is not reliable sometime and not hundred percent accurate but is only source to get keyword data. Should we rely on gwt completely? Is there any other reliable way?

    Reply

    Dana Loiz

    Hi Pramod,

    I am glad you liked the post.

    This article would better answer your question as to why Google did this: http://searchenginewatch.com/article/2297674/Google-Not-Provided-Keywords-10-Ways-to-Get-Organic-Search-Data

    It should also clear your worries about any spamming activities being related to (not provided) :)

    As for the data from GWT, if you are not willing to run AdWords campaigns and retrieve keywords from there, I’d say GWT is the most reliable keyword data we have at this moment.

    Reply

    karan

    Hi Dana

    Thanks for the article, it is little difficult for seo’s to analyze their source of traffic after these analytic update, i just linked my analytical account to webmaster tool hope this will work.

    Reply

    Dana Loiz

    Hey Karan,

    Thank you for your comment.

    Linking GWT with GA is a great start, indeed. However, please be sure to understand that the data from GWT will not replace the (not provided) item in your GA reports. The reports from GA will remain just as before. The only advantage linking the two accounts offers is that you can access the Search Queries report from GWT, within GA.

    To find the keyword data from GWT in your GA account, you need to go to the Acquisition menu of GA and select Search Engine Optimization and then Queries.

    Let me know if you need any further help with this, I’d be happy to help :)

    Reply

    Noufal Binu

    Hi Dana Loiz, You are mentioned 100% correct on my Google analytics account Organic keywords only show “not-provided” and little old data s! BTW i have also “GW” account linked with “GA” But it is not working. it is Really Google going to Harsh and Delirious way, i think Google Trying To Premium users only this Features Accessible and cumming days Google GA Features Axe One by one? :) Now i am try to another alternative service in organic keyword. this article lot of helped Thanks dana.

    Reply

    Dana Loiz

    Thanks for your comment, Noufal. I am glad you liked the post.

    Can you give me more details about what went wrong when linking GWT with GA? I’d be happy to help you sort things out.

    Reply

    Noufal Binu

    Thanks kindly Reply, It is OK also Get in there.

    Reply

    Pritam Nagrale

    Google is taking every possible measure to demotivate bloggers & SEO industry. Their main focus in promoting AdWords & not providing quality search to the people. Just because of few spammers, they are killing teh all means of providing quality content by people like us. We can’t write good stuff, if we don”t see any results.
    What is the meaning of quality search, if they are not allowing small businesses to rank on first page through SEO but through AdWords. A normal person does not have idea if that is the ad.

    Reply

    Noufal Binu

    I agree with Pritam, Google is Top One Search Engine but Recently Google SEO is not a right way for me, Expect Google Love Quality Contents and update SEO friendly in the next time :)

    Reply

    2BubbleBlog

    Nothing forbid you to fetch the referrer by setting a php function. Of course, it has to be placed in the footer or header of the pages of your customer, it’s the only annoying part but, I do it for years on my sites and it still works encrypted or not. You can even add a counter for similar queries to add extra information ;)

    Reply

    Amey

    Thanks Dana for your valuable article!!! Is there any way by which we can decrypt the search queries?

    Reply

    Dana Loiz

    Thank you, Amey for you nice words. I am glad to hear you found the article useful.

    Unfortunately no software can retrieve the keyword information with the secure search. This article explains in more detail how the secure system works and why the data is completely lost:

    http://searchenginewatch.com/article/2297674/Google-Not-Provided-Keywords-10-Ways-to-Get-Organic-Search-Data

    Reply

    Leave a Comment

    Previous post:

    Next post:

    `