Keyword Analysis in a world of 100% ‘Not Provided’

The percentage of organic search traffic that is being reported as ‘not provided’ is increasing for every website. Driven by changes to browser defaults as well as Google’s campaigns to get users logged in, most sites have lost this keyword data for considerably more than 10% of search visitors, and for more than 60% in some cases. (You can read Mike Pantoliano’s analysis on the Distilled blog.) Within 12-18 months, I wouldn’t be surprised to see most sites with 90% of their Google traffic being ‘Not provided’ and it could be 100% for some. Indeed, Google could turn all this data off overnight, at anytime, if they wanted to.

Some people have suggested extrapolating data to fill in the blanks—but this simply doesn’t work for now, and won’t even be an option when there is no data. It’s time to admit that we won’t have that data in our analytics packages for much longer, and to plan what to do once it’s gone. Before looking for other data sources, you should consider what you’re using keyword data for at the moment. Only then can you design a strategy for replacing what you’ve lost.

To help your planning, I’ve listed three particularly popular uses of keyword data, along with recommendations for actions you can take right now, and ideas for how to deal in a world of ‘not provided’.

1 - Assessing the effectiveness of SEO efforts and individual campaigns

This is one of the major reasons that people keep an eye on their keyword data.

From the perspective of overall SEO success, you can cope just fine without being able to see the breakdown between keywords. If the volume of traffic you’re receiving through organic search is going ‘up and to the right’ with appropriate velocity, then you have nothing to worry about.

The difficulty comes when trying to quantify the results from putting effort into ranking well for a specific keyword. This is something that we generally advise against focusing on too much anyway.

There are two ways to approach this—the first being to look at the amount of organic traffic coming in to a particular page. By focusing on the amount of traffic the page received from organic search (or persuading your boss/client to do the same) it’s quite possible that you’ll be able to reframe the conversation to focus on ‘the right things’. For example, if you were working to get a page ranking well for ’football’, but it’s drawing in loads of traffic, then you really shouldn’t mind if that’s actually coming in on terms like ’football scores’ and ’football news’. In fact, this actually gets to the crux of what we will have to deal with from an analytics perspective: at the most granular level, organic traffic to a particular page will the metric we’ll have to monitor. It’s time to start getting excited about the aggregate amount of traffic to a particular page again!

The alternative way to assess campaign impact will be to review ranking positions. There are a pretty big selection of tools available that deliver ranking reports, etc. At Distilled, we’ve used Authority Labs, SEOmoz and SearchMetrics for various parts of the puzzle. However, most of the available tools are stuck in a paradigm where there is just ’one true ranking position’ for each URL/keyword combination. Because of all the personalization in Google’s search results (a development that is unrelated to ‘not provided’), it’s only really fair to actually say that a page was ranked at a certain position for some of the time and another position for some of the time. One very accessible tool that does this is Google’s own Webmaster Tools. For example, the table below shows how rankings for a particular page have varied a fair amount, mostly due to personalization.

 

However, because Webmaster Tools won’t store your data indefinitely (in the way that GA or other Analytics packages will) you’ll need to track this for yourself, in order to see improvements over time. Grab the table data on a monthly basis, bring it into a spreadsheet or database and cut it anyway you wish.

If you’re Python savvy, there is also a way to automate the download process on a monthly or weekly basis. Remember: even though we still have keyword data available for now, it’s likely that you’ll want historic data at some point in the future. If you start collecting this now, you’ll have some historic numbers to look back on when there’s nothing left in your analytics.

2 - Comparing branded to non-branded traffic

A popular analysis of keyword data is to compare the amount of branded and non-branded traffic. It also offers insights into particular changes; for example, if organic traffic to a brand’s home page increases, it may be because they are continuing to rank better for non-branded terms, or it may simply be because people are searching for the branded terms more often.

An Inbound Marketer would argue that it doesn’t really matter what the breakdown is, as long as more people are arriving at the site. However, this might not satisfy either an SEO Director or a Brand Marketer who would both like to know how effective each of them have been.

The simplest solution here is to step up the funnel, and take a look at search volume for the terms you’re considering. The example below uses Google Insights, to show the volume of branded search for a particular company. In this case, the people responsible for brand building / marketing should be very proud!

3 - Conversion Rate Optimization

Many websites run CRO tests, to improve the performance of the site. While this has often relied on optimizing the conversion rate for a particular entry keyword or group of keywords, we can now only consider the conversion rate of individual landing pages. This simply means we’ll be focusing on the right things.

There’s potentially a little loss in data fidelity here; a page could theoretically receive traffic for one high-converting and one low-converting term. However, as long as you begin by measuring the CR for people landing on that particular page and work to improve that, you’ll be working on the right thing. In this situation, you might consider creating an alternate page to better target the poorly-converting term, attracting searchers to that new page and working separately to CRO that page. However, if that was a task worth undertaking, it should already have been done—there’d be no reason to wait until keyword data has disappeared.

4 - Finding content ideas

Analysis of a site’s keywords has often been a tactic for generating ideas about what other content to add to the site.

One way of doing this is to look for pages that are receiving traffic for mid- to long-tail keywords on pages that don’t really target that keyword, and don’t have a very good ranking position. In this case, you can create new content to better target the term, aim for a higher ranking and more traffic. Without keyword data this analysis becomes more difficult, but (again) Webmaster Central could help: you could look for phrases that are of medium length where the site ranks on the second page, and then judge whether the page really is well targeted to the term or not.

Another tactic for generating content ideas has been to look for groups of keywords that have low engagement / a high bounce rate. This suggests that users aren’t finding what they wanted on your site, and you can work to improve that. One alternate data source for doing this in the future could be analysis of your internal search data (which is pretty trivial to set up in Google Analytics). Another option would be to look elsewhere for tools to help with keyword/content research; I’m  a big fan of Ubersuggest.

Summary

There are myriad reasons that people use their site’s keyword data. Hopefully I’ve demonstrated that many of these functions can be replaced—either by using Google Webmaster Central (which will continue to be invested in over the next 12 months) or other tools and techniques. In a broad sense, the change may encourage people to focus on the right metrics and activities more often—and even implement best practice that’s been overlooked so far.

As an aside—and this is a controversial perspective within the SEO industry—I broadly support Google in this move, and believe the way keyword data was shared until now left some pretty serious privacy holes. But that’s a topic for another post!

With that in mind, I’m really committed to not letting this change be a huge problem for people. If you have a particular analysis of keyword data that you get value from, please tell me about it in the comments below. I’ll do my best to suggest alternatives that will be useful when we finally see the end of keyword data.

Rob Ousbey

Rob Ousbey

Rob joined Distilled’s London office in 2008 as an SEO Consultant. Over the years, he’s developed and executed SEO strategy for clients from small businesses to large organizations, and managed Distilled’s Reputation Management projects, where he’s...   read more

Get blog posts via email

15 Comments

  1. Will Critchlow

    Interested in your thoughts about :

    Diagnosing big swings in traffic (in either direction).

    Drops could be spotted if rank tracking was possible across all keywords I guess.

    WMC doesn't update fast enough to catch big fluctuations does it?

    And... similar to your cro one : there will be no way to filter out weird rankings that distort conversion rate (e.g. When you ranked for Halloween costumes or Paddy ranked for Brighton briefly).

    reply >
    • Similar to Will's last comment..

      With my niche in local, we tend to get some customers ranking for national terms which don't convert. It seems like it would be extremely difficult to account for these situations.

      I guess we should all start preparing for the inevitable 100% not provided :/ . Thanks for sharing!

  2. 4 really connected with me. This is a tactic I use a lot, and along with the others you outlined it will require a slightly different process going forward.

    Thanks for writing this post Rob, it seems like 90-95% Not Provided is an inevitable situation that SEOs are trying to ignore. This blog is once again ahead of the game.

    reply >
  3. Nice write up Rob!

    For me I just use landing pages to target specific keywords the client wants to rank for and then look at the organic traffic hitting that page, along with what's still public. Sure, it's not as accurate as the data we are used to but I've found it's enough to justify the effort.

    As for Google making this move based on privacy, I'm not convinced, they've taken a few big swings at SEO in recent time, that being one of them...

    reply >
  4. Dark times are coming. Google is making a war on SEO (and this is nicely described in http://www.wordstream.com/articles/google-ads). It seems that Wall Street demand for ever higher profits is making them forget their motto (think about the user, don't be evil and whatever else).

    My suggestion is:
    - get the data from AdWords and build awesome content around top converting keywords
    - learn and use CRO
    - create awesome customer experience so you are not depending on SEO

    Cheers

    reply >
  5. Jon

    It's a sad state of affairs. Can you see Google doing a u-turn at any point?

    reply >
  6. PPC or even YouTube analytics might be further ways to identify new keyword opportunities if you need them. Awesome article Rob.

    reply >
  7. Thanks Rob. I think you're spot on there.

    Last year I did some work for a mental health charity where keyword analysis was very valuable. Being a major charity they had a very strong domain authority and with the right placement on the site, their content would rank well. The challenge was getting the right content to rank for the right keywords.

    Keyword analysis in GA showed that many visitors in need of information and support for mental health issues were landing on the wrong page and bouncing. Without the keyword data, we wouldn't have been able to identify this and improve the site.

    What would you say is the solution for charities hoping to optimise their sites now?

    reply >
  8. This obviously contradicts the whole meaning of the 'Internet' - Keep it free from all influences and make it available to all! and Yet Google is dictating who holds the power and information!!!! I think long term this will be Googles downfall regarding SEO!

    reply >
  9. I remember Matt Cutts specifically mentioned in one of his videos as to how to build an authority website. He said that we can start by picking up a relatively easier term, start weaving our website around that and keep moving on to things which are more competition intensifies. I have personally given up on G analytics for my new websites. I miss the wonder wheel though, that was a great tool for professionals.

    The article is great, especially on the facet of leveraging the options and functions of GWT.

    reply >
  10. This is a great article, thanks for some practical tips! Data is always changing, and we'll always need to adjust our measurement.

    But I think there is a major measurement piece missing: the multi-channel measurement of organic search and it's upper-funnel effect on conversion and revenue.

    For a long time marketers have attributed conversion to the last click because that's the only data they had. But now we have multi-channel , multi-touch measurement, which makes it easier to understand how different channels, like organic search, play a role in the customer's decision making process.

    If we really want to show the value of organic search we'll focus on how organic drives direct conversions AND assists conversions.

    reply >
  11. Great article Rob!

    I've tried for several years to have clients focus on over-all relevant organic traffic growth rather than fixating on specific keywords as a measure of success.

    With a recent client the % of 'not provided' keywords was over 50% and I found the only way to measure and report to the client was to focus on total organic traffic and show the quality and relevancy based on KPI's like bounce-rate, number of page views and ultimately conversions.

    reply >
  12. Jennifer Wong

    Rob,

    Thanks for sharing recommendations for actions us marketers can take to deal deal with the increasing percentage of ‘not provided’ data.

    We recently conducted a study that revealed that Google’s “not provided” keywords have increased by 171% over the last 12 months.

    http://www.optify.net/press-releases/optify-study-googles-not-provided-rises-to-almost-40-of-organic-traffic-for-b2b-sites

    Jennifer
    @Jenerationy

    reply >
  13. Great article and practical advice, thanks Rob!

    reply >

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>