Keyword Analysis in a world of 100% ‘Not Provided’

The percentage of organic search traffic that is being reported as 'not provided' is increasing for every website. Driven by changes to browser defaults as well as Google's campaigns to get users logged in, most sites have lost this keyword data for considerably more than 10% of search visitors, and for more than 60% in some cases. (You can read Mike Pantoliano's analysis on the Distilled blog.) Within 12-18 months, I wouldn't be surprised to see most sites with 90% of their Google traffic being 'Not provided' and it could be 100% for some. Indeed, Google could turn all this data off overnight, at anytime, if they wanted to.

Some people have suggested extrapolating data to fill in the blanks—but this simply doesn't work for now, and won't even be an option when there is no data. It's time to admit that we won't have that data in our analytics packages for much longer, and to plan what to do once it's gone. Before looking for other data sources, you should consider what you're using keyword data for at the moment. Only then can you design a strategy for replacing what you've lost.

To help your planning, I've listed three particularly popular uses of keyword data, along with recommendations for actions you can take right now, and ideas for how to deal in a world of 'not provided'.

1 - Assessing the effectiveness of SEO efforts and individual campaigns

This is one of the major reasons that people keep an eye on their keyword data.

From the perspective of overall SEO success, you can cope just fine without being able to see the breakdown between keywords. If the volume of traffic you're receiving through organic search is going 'up and to the right' with appropriate velocity, then you have nothing to worry about.

The difficulty comes when trying to quantify the results from putting effort into ranking well for a specific keyword. This is something that we generally advise against focusing on too much anyway.

There are two ways to approach this—the first being to look at the amount of organic traffic coming in to a particular page. By focusing on the amount of traffic the page received from organic search (or persuading your boss/client to do the same) it's quite possible that you'll be able to reframe the conversation to focus on 'the right things'. For example, if you were working to get a page ranking well for 'football', but it's drawing in loads of traffic, then you really shouldn't mind if that's actually coming in on terms like 'football scores' and 'football news'. In fact, this actually gets to the crux of what we will have to deal with from an analytics perspective: at the most granular level, organic traffic to a particular page will the metric we'll have to monitor. It's time to start getting excited about the aggregate amount of traffic to a particular page again!

The alternative way to assess campaign impact will be to review ranking positions. There are a pretty big selection of tools available that deliver ranking reports, etc. At Distilled, we've used Authority Labs, SEOmoz and SearchMetrics for various parts of the puzzle. However, most of the available tools are stuck in a paradigm where there is just 'one true ranking position' for each URL/keyword combination. Because of all the personalization in Google's search results (a development that is unrelated to 'not provided'), it's only really fair to actually say that a page was ranked at a certain position for some of the time and another position for some of the time. One very accessible tool that does this is Google's own Webmaster Tools. For example, the table below shows how rankings for a particular page have varied a fair amount, mostly due to personalization.


However, because Webmaster Tools won't store your data indefinitely (in the way that GA or other Analytics packages will) you'll need to track this for yourself, in order to see improvements over time. Grab the table data on a monthly basis, bring it into a spreadsheet or database and cut it anyway you wish.

If you're Python savvy, there is also a way to automate the download process on a monthly or weekly basis. Remember: even though we still have keyword data available for now, it's likely that you'll want historic data at some point in the future. If you start collecting this now, you'll have some historic numbers to look back on when there's nothing left in your analytics.

2 - Comparing branded to non-branded traffic

A popular analysis of keyword data is to compare the amount of branded and non-branded traffic. It also offers insights into particular changes; for example, if organic traffic to a brand's home page increases, it may be because they are continuing to rank better for non-branded terms, or it may simply be because people are searching for the branded terms more often.

An Inbound Marketer would argue that it doesn't really matter what the breakdown is, as long as more people are arriving at the site. However, this might not satisfy either an SEO Director or a Brand Marketer who would both like to know how effective each of them have been.

The simplest solution here is to step up the funnel, and take a look at search volume for the terms you're considering. The example below uses Google Insights, to show the volume of branded search for a particular company. In this case, the people responsible for brand building / marketing should be very proud!

3 - Conversion Rate Optimization

Many websites run CRO tests, to improve the performance of the site. While this has often relied on optimizing the conversion rate for a particular entry keyword or group of keywords, we can now only consider the conversion rate of individual landing pages. This simply means we'll be focusing on the right things.

There's potentially a little loss in data fidelity here; a page could theoretically receive traffic for one high-converting and one low-converting term. However, as long as you begin by measuring the CR for people landing on that particular page and work to improve that, you'll be working on the right thing. In this situation, you might consider creating an alternate page to better target the poorly-converting term, attracting searchers to that new page and working separately to CRO that page. However, if that was a task worth undertaking, it should already have been done—there'd be no reason to wait until keyword data has disappeared.

4 - Finding content ideas

Analysis of a site's keywords has often been a tactic for generating ideas about what other content to add to the site.

One way of doing this is to look for pages that are receiving traffic for mid- to long-tail keywords on pages that don't really target that keyword, and don't have a very good ranking position. In this case, you can create new content to better target the term, aim for a higher ranking and more traffic. Without keyword data this analysis becomes more difficult, but (again) Webmaster Central could help: you could look for phrases that are of medium length where the site ranks on the second page, and then judge whether the page really is well targeted to the term or not.

Another tactic for generating content ideas has been to look for groups of keywords that have low engagement / a high bounce rate. This suggests that users aren't finding what they wanted on your site, and you can work to improve that. One alternate data source for doing this in the future could be analysis of your internal search data (which is pretty trivial to set up in Google Analytics). Another option would be to look elsewhere for tools to help with keyword/content research; I'm  a big fan of Ubersuggest.


There are myriad reasons that people use their site's keyword data. Hopefully I've demonstrated that many of these functions can be replaced—either by using Google Webmaster Central (which will continue to be invested in over the next 12 months) or other tools and techniques. In a broad sense, the change may encourage people to focus on the right metrics and activities more often—and even implement best practice that's been overlooked so far.

As an aside—and this is a controversial perspective within the SEO industry—I broadly support Google in this move, and believe the way keyword data was shared until now left some pretty serious privacy holes. But that's a topic for another post!

With that in mind, I'm really committed to not letting this change be a huge problem for people. If you have a particular analysis of keyword data that you get value from, please tell me about it in the comments below. I'll do my best to suggest alternatives that will be useful when we finally see the end of keyword data.

Get blog posts via email