Google Loses Twitter Stream, Wins the Search Game (Opinion)

Google disabled their Realtime search over the weekend because their contract with Twitter, dating back to 2009, has expired. Based off of a preliminary study yesterday by The Wizard of Moz, Google can no longer see and index based off of tweets. Some may argue that this is a bad thing, that the freshness index is going to suffer, and that Google is relying too much on Plus because Circles are not sustainable.

I disagree. Let me explain.

*Before I explain, I must leave some caveats. No one has had the time to robustly test how well or not tweets are being indexed (except for Rand, and we don’t have findings yet beyond this, ironically posted on Twitter), and as far as I know no one has run tests by just posting a link to a new article on Plus and seeing how long it takes to appear in the SERPs (though I am going to do it soon). Since this post does not have much data to work from, let’s put on our philosopher hats and think a bit.*

Why Google Indexed Twitter

Google began indexing Twitter streams back in 2009 when they realized the power of social. With the increasing need for and usage of personalized search, it made sense to index tweets to find new articles (and the Caffeine index update helped with this) to provide results based off of your social circle, which supposedly will contain better results for your individual needs.

This was great move on Google’s part I think. Search quality has improved, users can find the target of their searches more easily now, and we see connections from our Twitter connections (which I am still seeing today upon writing this post).

Over time, however, Twitter has become filled with spam accounts and followers (as brilliantly rendered through this cartoon (this link points to Mashable) from Hubspot), necessitating tweets like this from Twitter because of a tweet’s power to help search rankings. According to the following data (which is a bit old at this point), Twitter is fighting the battle well, but it also shows the prevalence of spam accounts to necessitate this battle:

One can argue that Author authority was a factor in deciding if tweets should be counted for rankings or not (and I did on SEOmoz back in May), but I also have a hard time believing that spam accounts did not affect rankings at all.

Enter Google+

When Google announced Plus last week, I started wondering how Plus, Twitter, and Facebook were all going to be worked together into the SERPs. It now seems that we do not need to worry about this.

Google not indexing tweets anymore is a brilliant move on their part, I think. Think about it: when we posted links on Twitter to help with speed of indexing and rankings, Google was forced to rely on a third party to gather their data. They had already run afoul of Facebook, and thus could not personalize searches or populate rankings because of Shares, which was also proven using science on SEOmoz. This deal went to Bing.

Google has historically hated to rely on third party services to discover pages and sites, so it only makes sense that Google would develop Plus to begin building their own link graph. This way, should a deal with Twitter go bad or not be renewed (like just happened), Google’s Realtime search would not be completely dead in the water and they would not be forced to accept a bum deal in order to provide freshness data.

The Potential of Plus

I like Plus for a number of reasons. Google developing Plus is a  huge step forward in my mind because they have realized the power of social, the effectiveness of personalized results, and their need to be autonomous because of the fallibility of business deals.

First, let’s think: when Google had to index Twitter streams to find new links, they used a third party. With Plus, when we post new articles into our Plus streams (Circles? I’m still getting my head around this new lexicon), we are effectively telling Google directly about new content. Also, we are telling only those we are connected to about our articles. This is different from both Facebook and Twitter, which blast out links and information. With Plus, we have more controlled sharing.

What about Brands? Won’t the lack of tweets hurt Google’s Realtime index?

What about brands?

For the past couple of years, we as SEOs have been encouraging our clients to get social, to make a Twitter presence and to build Facebook pages to drive more business, and traffic to their sites. The Twitter presence has been shown to affect rankings positively, and Facebook pages are simply a good idea because of the mass of people on Facebook (though I’d love to hear a counter argument about why Facebook pages are a bad idea). Essentially, the more followers an account has, the more likely it is to be authoritative, authentic, and influential (just think of the number of spam accounts you have seen with zero followers).

I think that with the upcoming release of pages for Plus, brands will need to build out pages on Plus as well. We should probably encourage them to do so in order to connect with that audience. As Plus grows, these businesses will be able to increasingly build relationships with their clientele (since Plus allows comments on posts, though these can be disabled if the publisher desires), and if links shared on Plus directly affect rankings and the link graph, then we would be wise to heed this advice and to direct clients to begin using Plus when it is available.

What about realtime search? Won’t Google’s freshness index suffer?

Short term, I think Google’s freshness index will suffer. My guess is that Google planned to renew their contract with Twitter at least for a while, in order to keep the freshness index up-to-date while they built out Plus. If Twitter is playing hardball, then Google’s hand has been forced and Plus will be pushed forward more quickly. If this is the case, then Google’s freshness index will suffer in the short-term, but not long enough to really hurt the search giant to such an extent that another engine, like Bing or Blekko, could take over. If you’re looking for a fun game to play to see which results you like more, check out this page and insert any search query followed by “/monte” to play.

Google’s freshness index will suffer for the time being, yes. If we take a longer term look at this area, though, I think that Google will come out stronger because they are now gathering their own data and will be able to more easily discover content. Losing Twitter data is a blow, to be sure, but not a game-changer or the final whistle.

We’d love to hear your thoughts and opinions in the comments!

Get blog posts via email

20 Comments

  1. SeoSteven

    Great article. If twitter is playing hardball they are probably going to be the only ones that lose. Long term at least.

    reply >
  2. Interestingly this is a rare ocurrence of Google stepping out of a market that they could otherwise have dominated. For brand monitoring software this is a blessing, as realtime search could do a lot of brand searches for free thus making some of these tools obsolete.

    If they had carried on, this could have been like Google Analytics making paid analytics tools pointless all over again.

    reply >
    • John Doherty

      Interesting thoughts Mike. Do you mean with Google stepping out of the Realtime game? According to The Next Web, Google is moving to integrate Plus data into the Realtime slot (http://tnw.co/mlQuER). Do you think the data will become as good as it is now, if not better?

  3. Integrating plus data is great, but it doesn't stop brands wanting to know what people are saying about them on Twitter. The removal of this from Google, restricts them from being a complete real time package - as now it's just real time in Google+.

    Google probably don't care, and have no interest in becoming a brand monitoring tool, but it's certainly got to leave a few smiles on the faces of those who do.

    reply >
    • John Doherty

      Ah yes, very true. Thanks for clarifying your point! I wasn't quite sure what you were saying about it being a brand-monitoring tool. Maybe there's a market that needs filling?

  4. SeoSteven

    That's true Mike, but Realtime was already missing Facebook data, making it a less than perfect tool before they lost Twitter.

    reply >
  5. Definitely agreed that the longer term play is the more interesting development here. Google are positioning themselves to know more about how people use the web than anyone out there. Being able to connect with and categorize folks, integrating G+ into sites easily, and let's not forget about the other G products that can tie a lot of usage and demo data together (Wallet anyone?) into a nice, useful, profitable set of info. I think if execution continues to be good on G+ (and i think it does need to iterate more to appeal to wider user base) then the algo can lean on G+ data VERY heavily sometime in the future. Which could either make one sigh with relief or shudder in horror.

    Even bigger picture: does anyone think it's beyond the realm of possibility that this can take G a step closer to bridging the gap between offline n online marketing intel? If G owns that data and creates the products that can track ROI no matter where transactions occur, why would anyone need anything else to do business?

    reply >
    • John Doherty

      Dave - Interesting point of view yourself! I do not think it is beyond the realm of possibility that Google is creating an environment in which they can be the go-to for business and bridge the gap between online and offline marketing. I think we are going to see more and more developments, especially since Google now has Deals and Wallet and Swipe is being adopted at a decent rate. We're going to see a lot more shifts soon, I think.

  6. I'm quite worried about this from a bunch of angles:

    • Google's access to Twitter has improved their freshness and relevancy around recent topics a ton. If they've lost that, search quality gets worse.

    • Fighting Facebook alone is pretty hard; fighting Microsoft and Facebook is harder. Fighting Facebook, Microsoft & Twitter at the same time? Is that really necessary? If they could get along with Twitter, at least they'd have an ally in the social world.

    • Twitter has big revenues coming from Google; it would suck if we saw Twitter growth trajectory slow because Google's millions aren't pouring in for this partnership anymore

    • As a marketer, it sucks if the value that Twitter held for boosting rankings/indexing/freshness/awareness of new URLs in Google is gone. Twitter is a great marketing tool, and the signal to noise is really high, so I'd hate to see Google revert to using the messier link graph exclusively :(

    Here's to hoping it's resolved quickly!

    reply >
    • John Doherty

      Rand -

      Thank you for the comment! Can I respond point-by-point:

      1) Agreed that Twitter data has improved Google's freshness and relevancy. But what if Google is now going to use the data from Plus instead? This could increase Plus usage and adoption to a broader audience.

      2) Agreed. Seems that Google has either an inferiority complex or a take over the world mentality. They do need a friend. Part of me wants to say that Google has possibly made a huge stumble by not renewing the Twitter contract, if they even had the option to do so.

      3) Also agreed. Twitter is innovating pretty quickly, and is a much-loved tool by a lot of people. This would be sad.

      4) Also agreed, but I think that if Google (and this seems to be a big IF, since I posted this article to Plus and it's still not indexed, 90mins in) is going to use links posted to Plus for these indexing purposes instead of Twitter, I'm not convinced they'll have to revert to just using the link graph. I could also not understand well enough how the link graph works...

      I'm excited to see how this thing plays out.

  7. Pulling my head right out of the search marketing maelstrom I can honestly say that Google's indexation of Twitter had almost no impact on my non marketing use of the search engine. At all.

    As such, while it might be bad for the marketers, I'm not sure the average searcher on the street is really going to notice or care that they've gone.

    Does anyone know any non search marketing people that are concerned about this?

    reply >
  8. Agreed with Rand on this. I don't think it wouldn't be good for a lot of people to diminish Twitter (with less money/partners), worse search quality. Also I would like to see Google make a dent in Facebook monopoly with business Pages :)
    Hoping this plays out well for Twitter.

    reply >
    • John Doherty

      Samuel -

      Thanks for the comment! I definitely agree in the short term. I guess we'll have to see what awaits for the long term! I'd also like to see Google make a dent in the Facebook monopoly, and for Twitter to keep innovating well.

  9. Why cut the timing so close, though? Google can afford a few more months of the feed, I'm guessing. They've now banked everything on Google+ when it's barely released and closed to invites. Given their many product failures over the last 2-3 years, I really wonder how smart this is. I like Google+ for now, but it's going to be hard to justify using 3 major social media platforms. When the novelty wears off, this could all fall flat, and Google's real-time relevance will suffer a lot.

    Enjoyed your analysis, BTW - just debating Google's logic :)

    reply >
    • John Doherty

      Thanks Pete! Totally agree, I second-guess the wisdom of the timing and putting so much faith in Plus. I do wonder if they had their hand forced, and I'm interested to see how it plays out over the next 3-6 months!

      If Plus falls flat, which is a possibility, real-time relevance will be flat and bad for Google. Then they'll really be in a tight spot, because they'll need Twitter data again, and if Twitter's smart, they'll charge something exorbitant for Google to get it.

  10. Google wont have to rely on their own link graph for URL discovery, but that discovery could be slowed a bit. So realtime URL discovery might suffer short term, but honestly, that's a small loss when longer term quality signals can assure that those short term URL bursts were truly legit.

    Also, unless twitter denies access to the Google bots, you can see via this example search string using PBS.org [ PBS.org inurl:status site:twitter.com ] that Google will still be able to identify newly tweeted URLs, just not as "newly" :) . But...it would seem to me that even though G+ is new, if a URL is of such importance or urgency or value that it's getting tweeted, then someone somewhere with a G+ account should have G+'d the URL directly or plussed it somewhere else somewhere along the lines. I also can't help but think that the third party social sharing sites are going to make it easy to tweet and G+ a URL at the same time.

    Prior to Google inserting the realtime tweetstweam into the SERPS, none of this was such a big deal. Google may no longer have instant access to the URL firehose from Twitter, but y'know, quality URLs find the surface, sooner or later.

    reply >
  11. Although search does appear bleak without the Twitter deal, I could see the possible integration with Twitter feeds on Google+. I've already got them showing in Linkedin, why not my Google profile?
    Perhaps Bing is offering an enormous amount to Twitter? Facebook, Baidu... then Twitter? Microsoft could be boxing Google from all angles. Next-up, Windows Cloudbooks that behave more like iPhones with apps and direct access to everything except Google.

    reply >
  12. Ultimately, I think it's a bad idea to put all of their "real-time eggs" in one basket - whether it's Twitter, Google+ or the newest and greatest real-time, social sharing/networking site, should they be counting on a single source to supply the real-time index? Can we really say that Twitter is the ultimate source of real time info across all industries? Until they can develop a way to index the real-time web by using info from multiple sources, all working together, I think there is a long way to go to make their fresh index benefit all users in all industries.

    disclaimer I have a 2-week daughter at home and this was my first day back to the office, also I was up way too late stuck in traffic trying to get home after the local fireworks display, so I apologize if my comments lack clarity or common sense.

    reply >
  13. John Doherty

    Eric Ward -

    Thanks for the comment, and great information and thoughts here. I totally agree that G+ is now going to become a player when we think about sharing information.

    Also, I ran that query that you provided, and did not see any data newer than May. I wonder if they are just having a hard time indexing now, and are not going to remove back data from the index? It will be interesting to see if the numbers of indexed tweets decreases over the next few days or weeks.

    I also agree that quality URLs find the surface sooner or later. It is a blow to SEOs, however, that tweets seem to no longer be affecting indexation speeds or rankings (once again, this is VERY preliminary research). This would be a huge blow to us, though not an insurmountable one by any means.

    Thanks for the comment!

    reply >
  14. Ian

    I dunno about this one. If Google starts biasing towards information collected on Google+, they're going away from their true value: Their ability to base rankings on the broadest-possible data set.

    Put another way: There's a lot of information flying around on Twitter, and Facebook, and yes, probably Google+ soon enough. If you only use one of those sources, or bias towards one of those sources, accuracy suffers. Plus, they'll be begging spammers to overwhelm Google+.

    Google's gotta be smarter than this. Per Dr. Pete, why wouldn't they wait a little while? They could test their data against Twitter's, etc..

    I suspect there's more going on here. Something forced Google to dump Twitter. I have no idea what it was, but my feeling is this wasn't totally voluntary on their part, regardless of their PR spin.

    reply >

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>