New Google Webmaster Tools Keyphrase Data is 70% Useless

I love data. I really do. So you’d think that the latest announcement from Google about the new organic traffic impression, ranking and CTR data would be right up my street!

Unfortunately, as with so many of the other data points in Google Webmaster Tools the data they give you is shoddy, innacurate and largely a waste of your time to analyse it. Why 70%? It’s a number I plucked out of thin air. See how annoying innacurate data is?

Here’s the stats for the Distilled site in GWT:

And here’s the Google Analytics data for the same time period:

See the discrepency? Let’s compare some of the top keyphrases (impression/clickthrough data is from GWT, visit data from GA):

I’m not going to go into a big analysis of the two sets of numbers because they’re clearly quite a way off still.

Two interesting things however that I draw from this:

1) Find long-tail keyphrases

The number of keyphrases that GWT is reporting is larger than GA but quite a way - analysing what these keyphrases are which you’re getting impressions for but not traffic might be interesting. I say might because in actual fact once you get into the long-tail GWT just starts to obscure the data with “<10”. Well 0 is less than 10 right Google?

2) Ranking Factors A-go-go

The REALLY interesting thing for me with this data is that GWT has a good track record of releasing data on things which affect your rankings. Crawl issues, duplicate content, site speed. You see the pattern? And now they’re releasing CTR as a big metric. I strongly believe that CTR either already is, or will soon, start to impact your ranking position. Looking at this figure as an aggregate across your whole site and comparing to similar sites in your industry I think it would give Google quite a reasonable perspective on which sites are and are not “brands”. And if this is the case, then CLEARLY they’re not going to give you the exact data. So my view on this is that you should treat CTR seriously, but that you shouldn’t rely on the data in GWT too much (other than perhaps a benchmark over time?).

What’s everyone else’s opinon on this?

Note - a few things I think I should mention. Firstly, be careful about restricting your data to web search. Image search queries aren’t picked up by default within GA, you have to set up a special filter to grab these, so that might explain some discrepencies (I’ve taken that into account in the above data). Secondly, I’ve looked at a lot more data than just one site but I’m not sharing all of that here as most of them are client sites. Suffice to say I’ve yet to see anything accurate.

Get blog posts via email

29 Comments

  1. Interesting analysis, Tom.

    I think it'll be useful as something to show the client from a benchmarking/reference perspective, but I certainly won't be defining and strategies on the back of it...

    reply >
  2. Yeah - I think there's an initial wow factor with this tool but quite a sobering realisation soon after that all is not what it seems with some of these numbers. Local search / image search (google "BMW X5 Black") - it would be nice if Google could break all of this stuff down for us!

    There's going to be heaps of chat about this in the next few days. I'm starting a big KW research project next week, and I'm going to see if I can make use of this tool for that. Hopefully I'll be able to come to a firm decision on how best to use this data quite soon.

    Regardless of accuracy / innaccuracy and unknowns it's still pretty cool what Google are doing for Webmasters with this tool.

    reply >
  3. This on the face of it is a great feature but as I looked more at client sites and my own sites, the numbers are a bit hit and miss. Smaller sites seem fairly accurate in terms of number of clicks, large client sites are quite a way out.

    I'd agree that the thing to think about is that Google are taking CTR into account, I've long believed this to be the case which is why I recommend spending time improving META descriptions to try and get the click. Google rely on it heavily for Adwords so it makes sense for organics too.

    Although, Rand did a Whiteboard Friday a few weeks ago and has the opposite opinion :) He basically says its far too easy for people to manipulate for Google to use it as an important factor.

    http://moz.com/blog/whiteboard-friday-influence-of-usage-data

    He has a fair point that there are other things to concentrate on more, but I still think CTR is a factor to be looked at for SEO.

    reply >
  4. I'm looking forward to comparing impression data against AdWords / KWT data.

    Hands up who thinks it's going to be completely different! o/

    reply >
  5. Interesting. I've looked at it on my site and I'm generally seeing very good agreement between the two sources, with GWT clickthroughs running between +5% to +10% above reported visits, which is about what I would expect (given that they are different metrics and given the fact that some users will hit the back button before the GA code loads).

    For example, on one of our key terms I see 22,200 clickthroughs via GWT and c. 20,700 GA visits on that term.

    There are always huge issues getting metrics to make sense together - I've lost hours trying to reconcile different analytics figures...

    However, I do thin that this new information is really helpful, especially the CTR and ranking position breakdowns. I'll be using it to track qualitative trends and patterns rather than as a reliable quantitative measure.

    reply >
  6. Nice analysis.
    I believe as with everything related to Google, you take it with a pinch of salt. That said I also believe due to the fact that its just been released the data isn't going to be 'outrightly' right. It will take some time for it settle and be more accurate than inaccurate.
    That said, I'me loving the new look and all that date and I'll be having a play around it and see what more I can get out of it...

    reply >
  7. Hi Tom

    I am not sure why your data isn't accurate, mine is pretty much on the ball all things considered.

    Here are some factors I have noticed in your data

    The total seems to reflect the total for one landing page, not necessarily all landing pages
    For distilled you have sitelinks in the UK
    It is a navigational query for some
    Uniques vs Visits

    Certainly on a page that is next to useless without javascript turned on, and fully loaded, such as the one I have ranking for wordpress htaccess and related terms, I am finding WMT to report about 90% of the total reported by GA, across many variations.

    The difference could be somewhere in the muddle of repeat visits, clicks, views etc, but it certainly isn't as inaccurate as you portray here.

    So then I look at some much bigger data for a site with 7-8 figure traffic.

    Top search term is identified correctly, as are many but not neccessarily all others (I need to spend more time on this)

    The number of visits reported in Webmaster tools is a little low, showing only 70% of the reported traffic in GA.
    Also the numbers don't add up, as if I look at the totals for each ranking position it comes to less than the total reported.
    I had the opposite for htaccess on my site.

    However percentage of new visits on the page is 60% and for a long time I have suspected it to be some kind of bot traffic due to insanely low (single digit) bounce rate and low earnings.

    With all the different factors and comparing with other vastly different bounce rate and unique visitor data my conclusion is that GWT might be showing uniques.

    The number you are showing and I was looking at for GA is visits

    The click data I am looking at is in the 4-5 figure range for a single keyword.

    When I look at specific landing page data I see the problem of sampled data which affects the headline numbers.

    When you are seeing 96 visits ±66% or 53 visits ±89% then you realise the data in GA might be the inaccurate one, not GWT.

    reply >
  8. Hi Andy, thanks for the comment - certainly sharkseo was suggesting that unique visits is most likely what they're working on. The large traffic sites I've looked at don't get much better in terms of data, even looking at uniques, factoring in some % not loading GA etc.

    That said - I certainly agree that the GA data is not accurate either, it's something to bear in mind with this kind of analysis!

    reply >
  9. Hi Tom, yeah noticed some of the discrepancies myself, however I'm doing a bit of a manual job now of comparing this over about 300 clients keywords to see what the average CTR is based on position in the SERP's, I must say the data so far is shocking, 26% CTR when keywords are on 3rd Page + (That sort of data would make some SEO's think page 3 was an accomplishment :) )

    Anyway I'll give you an update when I have sufficient data! I'm about 10% of the way through (Based on the idea of plucking something out of the air :) )

    reply >
  10. @Neil Ha, yeah we're all clutching at straws here. Let me know what you find!

    reply >
  11. Rich

    At the brand I work for, we bid on our brand name and variations in adwords, and this new data doesn't match up with that. For instance for the core brand name, adwords has approx 38k searches in a period of time; webmaster tools has 49.5k.

    In webmaster tools however the 49.5k seems to be made up of 2x 22.2k searches (where we show in position 1&2 for the brand term for different pages), plus some others (sitelinks etc).

    So how many people are actually searching for us and what percentage are not coming to us after a brand search? I'd love to know but god knows how to find this and be happy with the data!

    reply >
  12. I always chime into the CTR debate ;)

    You make a very important point about CTR being used: the data would have to be very much aggregate: if Google were to trust CTR data, it would have to be data gathered over a long period and it would have to be for all of the site's impressions, or at least where a click took place on a site in the SERP.

    Done this way, it could mean something, but as everyone points out, you could build a beautiful little network where your credit cards page was "visited" over and over again by bots with unique c class IPs, or even for a range of terms. I'm not saying no one could build a network to game the long-tail, but I think most people would find their network-building efforts better spent on something else. Generating the right queries alone without drawing attention to the activity would be hard enough, and pretty pointless for a small ranking factor.

    Google trying to make sense of CTR data that is difficult to spam makes sense though, as it ties in relatively well with personalised search. Far more complicated and done for different reasons, but relevant if it can be analysed correctly.

    reply >
  13. I'm already missing the ability to export the rankings for these keywords. Sure you can export the keywords themselves, but not the rankings. That's a step backwards!

    reply >
  14. ugh...just what I need. Another voice in the "does CTR impact ranking". I'm thankful for your thoughtful analysis but RandFish at SEOMoz (whom I also respect) says the exact opposite on CTR.

    Now who do I believe?

    reply >
  15. @Vincent - I know! Difficult to know how to believe, frankly I'm not sure there is a right answer on this. I'm fairly sure it's part of the "brand" analysis that Google does but I've not got much evidence to back myself up with and I know there are compelling arguments in the other direction (from Rand and others). Still, improving CTR is a good thing regardless of if it impacts SEO or not, so like site speed it's worth doing anyway.

    reply >
  16. very good and informative post. I always had a doubt on CTR as an ranking metric, and i believe in that (may not be true). But expert opinion always differs. I think ppl should bother about improving CTR along with placement/ranks. It is the content/copywriting which could do wonders on organic CTRs...

    reply >
  17. Paul

    I'm seeing quite a difference between GA and GWT too Tom.

    For one reporting period (1 month), GWT shows 165,000 impressions, 40,500 clickthroughs and 8000 search queries (24.5% CTR if you're interested...).

    Compare this to GA for the same site and same reporting period and I get "Google sent 101,000 visits via 20,000 keywords".

    That's a huge discrepancy between the 101k and the 40k figures. Interesting to note that GA is seeing 20k keywords versus GWT's 8k.

    I'd love to use this as a measure of how well our SEO tweaking is going but the figures are so far off the ones in GA I'm a bit wary of them at the moment.

    reply >
  18. Interesting, and I agree that the data is flawed, looking at larger sites too many figures are rounded up or otherwise similar to be used as analytics.
    However.. I see great usage as a CTR tool, for example, if you've similar pages, ranking for similar queries, but one has a higher ctr, you can test changing that.
    That and the potential for future use means I see this as a 30% great tool. :)

    reply >
  19. @Rob - I think we are 100% in agreement then ;-)

    reply >
  20. Tim

    I agree that the new Webmaster tools are very confusing. But Analytics can be equally confusing.

    I have Web Analytics through my hosting account. It tracks every URL referrer and reports the search query and the number of page views. I find that is much more accurate information, since it comes from my hosting server itself. Furthermore, it lets me go directly to the actual search and see where my URL lists.

    One thing Google can't report is if someone goes to a site, bookmarks it, then returns later for another look. My server logs show this behavior, and I think it is important because it shows human interest (the whole point). Unfortunately, my logs can tell if a robot is doing the same thing.

    BTW, I find that Google only accounts for about 25% of my traffic. I get a lot from Yahoo, MSN (Bing) and others that Google obviously cannot report.

    reply >
  21. Hi Tom, nice to meet you @ SASCON, I talked with you about this blog post and that I had downloaded data for 2710 keyword and made a nice excel sheet of CTR and Google Position, finally put the blog post up, like to hear any comments:

    http://www.seomad.com/SEOBlog/google-organic-click-through-rate-ctr.html

    Cheers

    Neil

    reply >
  22. Yes I agree with you Tom, that the data GWT provides is inaccurate than the analytics.

    I am not sure about if CTR will have an effect on rankings in the future. There are around 14 ways of getting traffic from Google. At the moment GWT doesn't give this information. And if a website is ranking in page 4 (let's say for men shoes) but are appearing in Google's shopping result in Page 1, do you think this will increase their ranking in SERPs?

    There are other things but this is just one of many concerns I have by having CTR as a ranking factor.

    I don't see how that is going to work be honest. what do you think?

    reply >
  23. Nice post Tom, thanks (btw I've been saying google will soon factor in ctr for so long now - despite many disagreeing with me).

    Has anyone considered that the 'clicks' data formulated in GWT is merely the result of weighted mathematical equations? In other words, where you're ranking on the page combined with impressions or historical search volume and standard ctr's for various positions in the SERPs?

    I think of it as the traffic estimator in adwords.

    Thoughts?

    reply >
  24. I realise this post is old but I'm struggling to find any info on the newly updated (Oct 2010) WMT data.

    By this I mean the search queries % change data. Is this compared to the previous equivalent period (as the time range selected)?

    The ctr data is the main benefit of WMT, but IMO the overall query data is counter-productive for Google. Surely it places emphasis on the big (1,2 or 3) word phrases, which not everyone can realistically achieve. This strikes me similar to toolbar PageRank, insofar as they may regret ever giving access to this level of data!

    reply >
  25. Also, this new data implies that Marc's point above is wrong (that the CTR data is just a fixed equation).

    When I improve a title, I see the resulting CTR improvement..

    reply >
  26. Hi guys,

    I'm jumping on every discussion about this because I want answers from Google.

    My problem is that the click data is too good to be true. In other words, I am challenging the fact that I have nearly trebled a client's SEO traffic.

    I strongly suspect that the webmaster tools data is being "infiltrated" by paid traffic....

    reply >
  27. Bad news: http://googleblog.blogspot.com/2011/10/making-search-more-secure.html

    Search queries are going to be encrypted, but we'll still be able to have a view with Google Webmaster Tools. I consider WM Tools as quite unreliable though (some of the top queries mentionned there are totally absurd).

    What are we going to do now?

    reply >
  28. Completely agree, although whilst it's not completely useful for analytics it can provide clues as to potentially high volume terms which hadn't been considered. For example, I found that some terms were getting a high volume of impressions with an average position of > 10, which led me to believe that plenty of visits would be had for < 10. Sure enough, some testing with Adwords also found this was a great keyword to optimise for (Google Keyword Tool said it had less than 100 searches p/m - when really it had over 5,000 based on 40% CTR when we were at number one!). John

    reply >
  29. GWT data is really useless for me. For my site, some keywords have average position <10 but I couldnt see any of them in the first search result page.

    reply >

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>