Recover From Google’s Penguin Algorithm

I have been receiving a lot of questions recently—from clients and in other private forums—about the Penguin update and where to start in a Penguin analysis. So to set expectations off the bat, this is not going to be an advanced post of Penguin theory, but rather what to look for in your Analytics and Webmaster Tools to see where the drop occurred. This includes visualizations of factors that I have seen, and how to check for links that might be harming you.

We’ll discuss what Penguin is, how to diagnose a Penguin traffic drop, how to check the toolbar PageRank (TBPR) or Page Authority (PA) of links, how to check indexation of links quickly, and then give some overview strategies about how to approach different Penguin scenarios.

Let’s get started, shall we?

What Penguin Is

First, we need to explore what Penguin is (and is not) and how it manifests itself.

Penguin was first rolled out on April 24th (and reported about here by Danny Sullivan). It’s the “over-optimization” penalty that we had all been waiting on to drop since Matt Cutts seemed to talk about it in March at SXSW.

Think of Penguin as Panda’s link cousin. Panda targeted low-quality content onsite. Penguin seems to be targeting overly aggressive anchor text (both internally and externally), especially from low-quality sources. There has also been mention of Penguin targeting keyword stuffing, which keeps with the “over-optimization” theme.

...such as this from the geniuses over at SEOipsum

We have since found out that Penguin operates like Panda in that it rolls out in iterations (as evidenced by Penguin 1.1 rolling out in late May), with some sites reporting recovery and new sites being hit. This is a classic iteration algorithm, meaning even if you make changes to your site quickly after being hit, you have to wait until the next iteration (most likely) to see whether or not you made the right changes.

Since Penguin is an algorithm, not a penalty, it does not help to submit a reinclusion request (though you can submit your site using this form if you believe you did not deserve to be hit).

How to Diagnose Penguin

As mentioned above, Penguin manifests itself by targeting overly-aggressive anchor text, both externally and internally, especially over-optimized anchors on low quality sites (which is why WPMU was hit according to Ross Hudgens). We must be careful when diagnosing a Penguin hit, though, because Google also started deindexing free directories, especially directory networks, in mid-May. When trying to diagnose a traffic drop the following indicators should be looked at (in this order):
  • Does the drop in traffic coincide with one of the reported dates (so far April 24 and May 25)?
  • Is it a site-wide drop or does it seem to be keyword-specific?
  • Did you receive a notice in Webmaster Tools at a different time that could signify that it’s a penalty (as opposed to an algorithm)?
The best way to check these is to first check in Analytics (and then verify in Webmaster Tools based on keyword and overall site queries). I’m going to show you how to do both of these, or at least how I check what has dropped. We are going to use some Excel wizardry to do this, but just easy Pivot Tables and a few simple formulas that I will provide you.

Let’s look at a Penguin-ized site.

Diagnosis How-To

Remember, these are the links that Penguin loves to munch on :
  • On low quality sites (low PR)
  • Exact anchors
  • Over-optimized exact anchors
  • Too many exact anchors over branded terms
We are going to look at Analytics to determine when the date happened, to determine which keywords dropped, and then we’ll investigate using OSE/Majestic and Excel.

 Step 1 - Analytics

The first step, of course, is using Analytics to slice data around the dates that Penguin rolled out to determine if the drop happened around the same time as the updates. We’re going to use the following views:
  • From March 1 to June 9
  • From April 29-May 12th compared against April 8-21 (Penguin 1)
  • From May 27-June 9th compared against May 6-19 (Penguin 1.1)

Look at March 1-June 9

We want to look at March 1-June 9th to get a long-term view to see if a noticeable drop has occurred. Here is what you might see if a drop occurred around Penguin (notice April 24th is highlighted):

There is a pretty obvious drop there, even though the site is low-traffic. You can easily see the difference in traffic levels before and after April 24th.

Compare Traffic Levels from 2 weeks before and two weeks after

Now you should compare two weeks prior to Penguin 1 and two weeks post Penguin 1, allowing a few days of buffer on both sides to give the algorithm time to shake out. Here is what you might see:

Check out the drop in visits. In this case, it’s over 52% between before and after the update. A pretty clear drop-off!

Compare Two Weeks Before and After Penguin 1.1

Since Penguin 1.1 rolled out on May 25th, we can now also see if the site was hit again (or possibly regained some traffic because of efforts made). Here is a screenshot of a site that did not recover, and may have even been hit a bit again (for a variety of reasons):

“Alright”, you’re probably asking, “so my site got hit. I see that. Thanks. Now, what can I DO about it?”

Investigating WHAT Dropped

Now that we know there has been a drop, we’re going to investigate WHAT dropped. We’re going to use a combination of Analytics and link data, sourced from OpenSiteExplorer AND MajesticSEO (both if possible).

Step 2 - Which Keywords Dropped?

In Analytics, using Traffic Sources > Sources > Search > Organic, you can see the difference in traffic being driven from each keyword between time periods. Since you’ve been optimizing for your highest-value and traffic keywords, the keywords that drop the most should be near the top. Penguin seems to be a keyword specific algorithm, not a sitewide algorithm. Here is an example of what you might see (I have had to black out the specific keywords here for confidentiality purposes):

These keywords are where you start. So download them in CSV format.

Check in Webmaster Tools

Once you have seen which terms have dropped, I also recommend checking them in Webmaster Tools on a keyword level. To do this, go to Search Queries:

By adjusting the date range back as far as it can go (currently 90 days) you can see if there has been a drop according to Google as well:

Find the keyword that seem to have dropped on that page and click through to it. If you see something like this, then you know you have lost visibility for the term:

Step 3 - Download Anchor Text Data from OpenSiteExplorer and/or MajesticSEO

Next you need to download the anchor text distribution data available in OpenSiteExplorer and/or MajesticSEO, depending on which one you have full access to. I am going to use OpenSiteExplorer here.

What you need to download is the anchor text distribution from OpenSiteExplorer, located at http://www.opensiteexplorer.org/anchors?site=www.YOURDOMAIN.TLD. You’ll be looking at something like this (Distilled’s backlinks):

*Note* - these top anchor texts are the anchors you must give attention to. These are most likely the anchors that dropped, as well as the related keywords. If you’ve targeted, for example, [online colleges] with aggressive anchor text you will most likely see a drop around keywords like [best online colleges] and [cheap online colleges].

So by now we have:

  • Diagnosed that a drop occurred;
  • Seen which high-traffic keywords have dropped;
  • Downloaded Analytics data and OSE anchor text spread.

 Step 4 - Combine Data to Pull Out Learnings

What I’ve been doing now is combining the data in Excel. As you can see from the below Excel sheet, I’ve put both the Analytics data (anonymized) and the popular anchor text (also anonymized) onto one sheet for now:

One other thing that I do is get the percentage decrease back into my data, as unfortunately this does not download from Analytics (dear Avinash, can you make this happen please?)

What I did was use the following Excel formula, assuming visits to the keyword, by date range, are in column C:

=-(C9-C8)/C9

This doesn’t give you completely clean data, but it does show you the change in every other column (on the later date level). Here’s how it looks now (with conditional formatting applied):

Now we get to combine this data using a Pivot table to see if the drops match up to anchor text. We are going to really be looking at:

  • The drop;
  • # of LRDs;
  • # of links for that term (to look for potential sitewides to deal with first)
I am assuming that all of you know how to use Pivot Tables. If you do not, and you need some help learning them, I highly recommend either watching this video on Youtube or working through Lesson 5 on the Excel Guide for SEOs.

The goal with using this Pivot Table is to combine the keywords that have dropped with the # of linking root domains and number of links. You can now look for an abnormally large links/root domains ratio, or by knowing roughly the number of linking root domains your site possesses you can see where you are over-optimized with external links and need to go about changing anchor text or removing links.

Here is the configuration that I have used on the Pivot Table to get the data that I need:

Checking Link Over-Optimization

We’ve talked about how to diagnose the Penguin drop and tie it back to the specific keywords, but now we need to prioritize links to start investigating. My recommendation is to follow these guidelines:
  • Start with the biggest traffic drop (numbers, not percentage)
  • Check the TBPR/DA/PA of the site/page on which the links lives
  • For sitewides (based off of links)
  • Check indexation to see if Google has taken care of the link for you

Check DA/PA Distribution

Of course, start with your money-making terms. You want to check the distribution of TBPR/PA across these to see how good or bad it might be. To visualize the links to a specific page in order to see the DA spread of links, there are a few tools available:
  • Tom Anthony built a tool that he gave away on SEOmoz that can do this for you;
  • Dr Pete gave away the initial spreadsheet on SEOmoz to check Page Authority; and
  • I created a Domain Authority checker spreadsheet based off of Dr Pete’s that is the second in this post, or you can just download it directly HERE and go to the post to figure out how to use it. You’ll get a graph like this:

Are My Links Indexed?

As I said above, Google has also been deindexing directories. If you have been doing a lot of directory linkbuilding, especially low PR directories, many of them may be deindexed. I’ve seen sites with over 80% of their backlink profile deindexed, which was to blame for their traffic drop (which did not correlate with the time of Penguin, by the way).

To check indexation, I recommend using Neils Bosma’s SEO Tools for Excel. There is a nifty formula called =googlepagerank() that will tell you the TBPR of the URL where the link exists. Pro tip: -1 means that it is not cached.

It’s easy to use, but you can see it here on the sheet I used for the pivot table (this sheet is called ‘Links to check indexation’):

Obviously, the links that this tells you are deindexed do not need to be worried about. You can then subtract them from the totals and get the overall number of links left to remove to get back to a homeostasis of a balanced link profile (balanced against competitors, of course).

Checking indexation based on dates can also help you figure out if you were hit by directories being deindexed. This will be around May 15th.

How to Proceed

Of course, following all of this investigation, I often get asked how to proceed. And the unfortunate answer is:

The right answer depends on your specific situation

If you had a lot of directories pointing to deep pages, but good links to your homepage, you might be able to salvage your site. If you have a situation like WPMU, where they had a lot of links from a lot of low-quality domains that they had control of, you can turn those off quickly.

If you have a case where your homepage has a lot of exact anchors and few branded and random anchors, you need to make the decision of whether it is worth the cost to do a marketing campaign to get branded links, whether you should use Penguin as a reason to rebrand and move sites, or a combination of these.

I made you a spreadsheet to help out

I’ve been showing you screenshots from an Excel spreadsheet spread throughout this post. In the interesting of giving away helpful things, I want to give you that spreadsheet as well!

Download the spreadsheet


I’d love to get y’alls thoughts on any of this. If you’ve written a good post about diagnosing Penguin in order to put together an action plan, feel free to drop the link in the comments below.

Get blog posts via email

92 Comments

  1. Such a great article. Thanks for the deep analysis. Probably the most usefull ressource I have been able to read about Google Penguin.

    reply >
  2. Hi John,

    Great post! As someone that's been doing a lot of Penguin and Panda-related investigations lately, I am all too familiar with slicing and dicing analytics data for April and May. You do an excellent job describing the process to others.

    Currently, one of the real challenges with these investigations is distinguishing between Panda 3.5 (April 19), Penguin 1.0 (April 24), Panda 3.6 (April 27), and other anchor text-related updates that many believe went live (without an official announcement).

    If any of your readers are interested in learning more about the events leading up to the Penguin Update, I recently wrote this post: Everything We Know About the Penguin Update.

    Thanks again for your post and its step-by-step approach to analyzing a Penguin attack. I'm sure it will help a lot of people get a better handle on why they've lost Google traffic. Keep up the great work!

    -Steve (@webbstuff)

    reply >
    • John Doherty

      Hey Steve -

      Thanks for your comment and that link to your post! I totally agree that it's difficult to diagnose which penalty it might be. Though Penguin seems to be keyword-specific, whereas Panda is sitewide. And the directories being devalued happened on a different date. They're all pretty close, but I think enough different that you can figure it out with a little elbow grease, no?

      We'll really be in trouble when they start rolling out Penguin and Panda concurrently.

  3. Hi.

    A client is suffering the Penguin Effect, he had bad links pointing to the homepage, all whit the same anchor.
    Bad Bad Bad.

    Anyway we start working whit the long tail, and you know what ?... we descend 40 positions in a day for a landing page situated inside de domain...

    Could be that Penguin is an entire domain penalization ?.. it look like this.

    Matias

    reply >
    • John Doherty

      Is that landing page optimized for a keyword related to the one with all the links? I'm seeing it on a keyword and related-keyword basis, not sitewide...

  4. This is without no doubt the best article I have been reading about the Penguin update so far, thx!

    I have already spread it outthrough my danish network.

    And now I will take a look at that excel tools :)

    reply >
    • John Doherty

      Good stuff Kristian, and thanks for the kind words. I hope your Danish network gets value from it :-)

  5. Thanks for a great post as usual John, the explanations and the tools are all really helpful.

    -Rich

    reply >
  6. Excellent post. I have an e-commerce site that was trashed by Penguin. Because the site has few external backlinks I'm pretty sure one source of penguinization is optimized internal anchor text on category pages.

    For example, in the sidebar I have descriptive product category links by size, color etc. Possibly too descriptive and optimized:

    Foam Widgets by color
    - Blue Foam Widgets
    - Red Foam Widgets
    - Green Foam Widgets

    I have since changed the links to:

    Foam Widgets by color
    - Blue
    - Red
    - Green

    Well see how this change takes.

    reply >
    • John Doherty

      Please let us know how it turns out, Matt!

    • Sam

      I'm curious how that change works out for you. Our sites also have many internal links in the nav which are all optimized for the same KW the page is. We have a top nav, a left nav, and sometimes a few footer links (though I'm trying to remove them). It looks natural though and all of our sites do it, but only 2 or 3 sites have been hit by algorithm changes and others with the same set up are performing as well as always.

  7. John,
    Great analysis. You mention that maybe you should use "Penguin as a reason to rebrand and move sites", would you really recommend this for someone who truly has a site with great user experience, good content, good links, but also a bunch of bad links? I even heard Matt Cutts say that was an option, which made me wonder how this update is good for users if they cant see a site in the SERP's that is actually a good site, but just has some over optimized, bad, or just not google approved links.

    reply >
    • John Doherty

      Hey Sean -

      This is a very tough question that definitely should not be made in a vacuum of SEO - it involves so much more with the business. But from an SEO/Google perspective, yes. I would and actually have recommended this for someone with a great user experience and good content. The one difference between the scenario I've dealt with and yours is that the sites didn't have really any good links - maybe 2 or 3, of hundreds.

      Companies rebrand all the time. If you have a legit business, but have been doing crap SEO, you probably have an email list built up and relationships with customers. You can email them and let them know. And, since we are seeing that 301 redirects don't seem to be working on Penguin, you can leave up your old site for a while with a "We've Moved, Find us at domain.com" page.

      It's quite a tough decision, to be sure, but in some cases I do think it can be necessary.

      John

  8. HI John,
    Ok so this will be a longish response but I would really love you to take a moment to answer.

    I have read your post and I really appreciate the inherant complexities of what you are discussing. I am a small business owner doing my own SEO and have been doing pretty well applying what I read in SEO blogs and other resources. However recently I was affected by penguin and when I began to read into more and more posts I have come to realise that some of the blogs I had previously trusted have been really talking about what works and not necessarily what is recommended by google.

    The fact is this is my own fault and luckily we have only had a drop of 30% in turnover and now that I have removed a bunch of sites and guest blog posts we have recovered half of what we lost.

    I have basically decided to remove all external content that was self generated and start to build a completely squeaky clean site. I always thought I was but I am now seeing the effect of the small grey areas. I have also unsubscribed from all seo RSS feeds apart from distilled, moz and googles own offerings. The trouble is I am a little unsure about the idea of excessive anchor text with regard to internal linking.

    Lets say I sell watches and watch accessories (which I don't) and I am adding new listings to our site regularly 4-5 products per week. As additional content I am either blogging about the differences between new products or something like that. I also have a how to section showing how to maintain various brands or common issues like changing a watch band. The posts on blog and in the how to are quite meaty like 400-1000 words as a generalisation.
    My question is that how many internal links am I supposed to use for good long term seo (basically so I stay clean and avoid any future algorithmic issues).

    The anchor text is rarely the exact same for example "citizen watch battery" or "new watch battery" but over an average of the posts I would think that of the nine anchor text links there would be a couple of double ups and mostly long tail variation. To my eye and the reader the post is completely natural and informative I am a genuine expert in my field and have a lot of other companies following my repair advice.

    Thankyou in advance for your comment, I just want to get this right

    reply >
    • John Doherty

      Hi Alec -

      Thanks so much for your question man. I really hope I can do it justice in a blog comment response!

      First, I think it's smart to focus who you are listening to, but remember that we all talk about what works (otherwise our clients would leave us). But many of us who are labeled "whitehat" SEOs and marketers - and funny enough, none of us have ever really branded ourselves that way - talk about both what works and also what will keep you in Google's good graces. I also recommend that you follow SEER Interactive and SEOgadget.

      Now to your question about internal anchor text. That is a tough one, and honestly I haven't dealt with a site where I think this is an issue. I do know of a couple of great posts that you could use to see how your internal anchor text is working and where you might be over-optimized (for example, if you got hit for a term that you don't have external links for, the next step is looking at your internal linking). Here are some resources for you:

      http://www.searchenginejournal.com/visualizing-link-data-with-screaming-frog-and-excel-part-1/43641/

      You can adjust this one to internal anchors and visualize them:
      http://www.seerinteractive.com/blog/visualize-your-backlinks-with-google-fusion-tables

      In regards to how to think about internal linking, hopefully this doesn't sound too general, but think about what the user would expect to see, and how they would expect to see it. The issue with the internal linking plugins that seem to have caused issues with Penguin is that the linking is unnatural when every instance of a word (ie [citizen watches]) is linked to your Citizen watches page.

      So I can't give you exact "how many to use" numbers, unfortunately, but I hope this starts you in the right direction.

      John

  9. This is the article I was going to write last week! Never mind, looks you did a better job then what I would have done anyway :)

    reply >
  10. Seriously - WHO CARES.

    I'm so sick of Google and their ability to kill someone's income because THEY don't like what a website does. One of my close friends just lost 50 percent of her income on her sites, and all she does is write very good copy that's not duplicated anywhere else, and isn't keyword stuffed. She also writes all her own stuff and doesn't pay anyone for links. Yet Google slammed her site.

    Feck Google. We now concentrate all of our efforts onto Yahoo, Bing and DuckDuckGo. Google can kiss my happy a&&. :)

    reply >
    • John Doherty

      Seriously, DuckDuckGo for the win! Actually, I really want Wiredoo to take off :-)

      On a serious note though, you bring up some interesting points. It does seem unfair that Google has the ability to kill off a business, but they also have to have rules. Is it unfair that a community can ban a user for spamming them? Are all laws unfair then? Google is a business, and they have to keep it running. It annoys me sometimes as well, and I hate it when legit businesses are hurting.

      I'd love to see your friend's site and see what might be going on. It may be a case where she did not deserve to get hit, and she needs to get it in front of Google. Please feel free to email me details at john.doherty@distilled.net!

      John

  11. Hi John Doherty,
    Thanks for sharing this great post !
    To combat the Google Penguin, SEO process really needs to be evolved.The anchor text links should now be taken care of before posting.Besides this,keyword stuffing must be avoided.Proper tracking of your keywords and investigating the reason why is it dropping is also required.

    Thanks again
    Regards

    reply >
  12. George

    Best article on Penguin so far!!
    When I compare organic search traffic 2 weeks before and after the Update, I see a 16% decrease in traffic. When I check on keyword level, I see a 50% decrease in traffic driven by the top keyword. Unfortunately it's "(not provided)". Any reccomendations how to proceed in this case?
    Regards from Switzerland.

    reply >
    • John Doherty

      Oi, that's a tough one George. I think we're going to start talking about October 18th (the day [not provided] was announced) as a D-Day of sorts for data.

      Unfortunately, I don't think there's anything you can do here. Are there other keywords that have dropped too? Also, if it is an across-the-board drop, look at Panda instead of Penguin. Another iteration of Panda rolled out around the same time as Penguin, so check that out.

      John

  13. Really too much impressive article with deep analysis. Each and everything as you described with images of Google analytic fantastic. Now i can analyses or get the feedback with this useful guide and find the best results to improve my blog. I think it is one of the major part for every SEO professional because without analysis we can't do best or compete.

    Thanks for sharing for such a useful stuff

    reply >
  14. Nice Sharing :)

    reply >
  15. Hey John,

    It's surely one of the best articles I have read on Google Penguin Update.... And the best part is the excel spreadsheet you offered for the download...

    reply >
  16. This is excellent stuff.

    Just in case there are any UK users who are following the step-by-step so closely that they forget about the Queen, I'd remind them that the double bank holiday at the beginning of June may skew their traffic comparisons for the suggested periods...

    reply >
    • John Doherty

      Good call, Henry. Some of the updates have taken place on Friday too, which throws off numbers because of the weekends for some of us.

  17. Wow. That was ridiculously indepth, but I agreed with just about everything. Awesome post, thanks for sharing!

    @Bob Now don't feel sad Try your best and Post a New " Penguin-Strategies"
    We are waiting Keep it up :)

    reply >
  18. Wow, that's what I call an in deep article about Penguin! I am sure you spend a lot of time researching for this. Thanks for sharing.

    reply >
  19. John,

    Fantastic Post, appreciate your efforts. I have a couple questions:

    The unnatural links messages in webmaster tools, Matt Cutts has stated that these are manual actions (which would imply they were sent out manually, by a human), but in most cases these seem to correlate directly to Penguin Updates from what I have seen, with them appearing April 24th, June 8th etc. Do you think these could be generated by the Penguin Update or related to it? Are they being sent via the Google Algorithm or via manual web spam members? 700,000+ warnings is a large amount to be sent manually. I have discussed a couple cases with others where they have received multiple unnatural link warnings on the same day, this would lead me to believe that they are "not" manual actions, as why would after a manual action has happened would another manual action the same day? I have also heard about an unnatural links message being sent more than once to the same website each time a penguin update is run.
    Also, It seems that when a website gets an unnatural links message,the traffic has rarely dropped right away; it appears there is a time between 2-4 weeks when the manual action hits the website's traffic. What might be the purpose of this delay, are they giving you the opportunity to attempt to correct the situation? Perhaps the warning is algorithmic, and then the delay gives a site owner to correct the situation before the penalty drops (manually)? Or maybe if these warnings are related to the new methods the are using with Penguin, they assume that they will run the Penguin update between every 2-4 weeks, therefore it will re-inspect the back links algorithmically before a Google Web Spam team member re-reviews the website in question to see if they have corrected the issues prior to actually applying the manual action? If the warning is driven by an algorithm, then sending a reconsideration request would do no good, yet Google does suggest a reconsideration request in the unnatural links email. And before a penalty has even actually occurred. Any theories on all of this?
    Another point: Matt Cutts has stated that only 3% of the warnings were regarding unnatural links in first few months of the year, I find this nearly impossibly to believe based upon the volume of discussions and posts out there in regards to this? Do you have any thoughts about this?

    Thanks for your time, and appreciate your efforts to share your knowledge with the community.

    reply >
  20. Thanks! Very helpful article, especially the pro tip about -1 meaning the page isn't cached. I always wondered what the -1 meant.

    Best,
    Brad

    reply >
  21. Great stuff and I had funny playing with date based comparisons.

    Unfortunately, I still have no solid clues about why my website got hit. Traffic quartered on the 25th (Australia, 24th in the US). This is after filtering out other search engines and Google Image results which did not change.

    Keywords in backlinks aren't excessive but my backlink profile is probably over saturated by forum links due to all the work I do in them.

    I don't have many, if any de-indexed directory backlinks. When I did do directory submissions it was hand done by myself and I picked local/relevant ones.

    It was not specific keywords that dropped. It's across the board. I've even got closely related keywords to ones hit that are doing better.

    Traffic even dropped when I still rank at #1/2 in the SERPS, with an author image (Tested in several countries).

    I'm just sitting it out at the moment. Anything I do to try and fix things would be a wild guess and just a panic reaction. I'm open to suggestions and am happy to share data.

    Question: You stated internal and external anchor text but only analysed external. Can you provide more info on the internal stuff? Almost all of my internal links use the page heading or similar for the link text so if there is anything in that it may be a new clue.

    reply >
  22. Thanks a lot ! This is an excellent post for people hit by Penguin ! I was a little sceptical if my post Panda work was going on fine, and the date comparison showed that it was !! However I know I have submitted to many directories, and getting them all removed is gonna be an uphill battle. Oh well, atleast I know WHAT and HOW I can proceed :)

    reply >
  23. Hi John,

    Exceptionally awesome, Well mannered and well captured content !

    Every thing is in arranged form. It seems that you have done a lot of hard work before posting this complete strategy for penguin update ?

    Also i was looking for some kind of good tips that how could we recover from Google penguin update ? For example if we have done a lot of blog commenting and those pages of blog commenting get scraped or link farms then how can we manage to get back our website ?

    Blog commenting can only be deleted from webmaster and we can only Email him and ask him to delete the blog comment. What if he does not read the Email ?

    I am looking for your response.

    Thanks

    Saif

    reply >
  24. Excellent post John

    Google and its staff are great aren't they? They'll penalize your site but not give you the reason why they did it so you can fix the problem.

    It could be an innocent mistake of trying to be as relevant and helpful as possible but Google doesn't care.

    They'll take you out of the top 3 pages regardless of how helpful your content is based on an algorithm which is focused on supporting the big brands with high DA and paid SEO's building them links.

    It amazes me how many times i have seen forum questions (without any answers) rank 1,2,3 in Google SERPS.

    The reason for this is the domain authority is so powerful they'll take the top 3 spots ahead of real answers and helpful content that does provider a range of great answers to the question

    The people that work at Google really are doing a great job aren't they!

    Its really time to start using Bing as the results there are far superior. Probably because the people that work at Microsoft are quality staff.

    I've written my own post about why Google took back your traffic with some recovery tips of my own: http://wpsites.net/seo/3-reasons-why-the-google-penguin-update-stole-your-search-engine-traffic-recovery-tips/

    reply >
  25. very clear post about Google update and clear explanation with reference sites and tools , honestly this is awesome article thanks for sharing

    reply >
  26. Deep stuff but a simple message: Develop and optimize websites for people, not search engines. Since the Penguin update was set in stone, I've visited countless local SEO companies' websites just to see how they have responded. So many of them still have pointless, annoying links in the content (i.e. a link for "SEO Company 'City'" pointing to their homepage) with exact anchor text. What on Earth is that doing for the customer? So thanks, Distilled, for helping me stay above the local competition.

    reply >
  27. Thanks for this post,as usual post was informative and straight forward , and i hope it will be proved helpful too..

    reply >
  28. I've noticed quite a few sites that was hit by Penguin still haven't recovered. I think we won't see anything till the next update.

    reply >
  29. Hi John,

    Great information and well thought out. Only thing that puzzles me, are those 700k+ notices sent out to webmasters in GWT about inorganic links, Google may have been running these as "test" cases which Penguin were applied against. But the action against those sites/links were manual and a penalty, if you can prove to Google's Quality Team you have removed the majority of these inorganic links, the action against your urls will be revoked - in some cases a partial improvement. These manual actions seems to be running in conjunction with Penguin and not Panda as it mainly involves links and not content quality.

    Only other beef with Google - in the past I did article marketing and my articles are on thousands of sites, now the links in those articles are considered spam by Penguin, yet Google and site owners are still profiting from Adsense ads which are plastered around those articles. Isn't this a bit hypocritical of Google? Profiting from what it now calls web spam?

    Ask Google for help in removing those links and Google says it can't act for all webmasters... until or if they come out with that "disavow link tool" for GWT, we are out of luck.

    reply >
  30. Probably the most effective posting about diagnosing Penguin, thank you for sharing.

    From what I see there aren't any proven tactics or instructions to recover from Penguin. Many said Google wants to get rid of SEOs with these algorithm changes, but I think they are setting an advantage for professional SEOs that really understand their business, work and love what they do, and a slap to so called SEOs who believe setting up meta keywords and building backlinks with the keyword in the anchor text is everything you have to do to get rankings.

    reply >
  31. I own this site www.jobsforfreshers.in and it is badly hit by Penguin. The fall is site wide and happened exactly on 24th of April. We are not able to analyse the reason. Could you please analyse and guide us??

    reply >
  32. Hi John,

    Thanks for a great post! I'm hoping that I'm in the deindexed links bucket - I have to run a historical report before I can be sure but fingers crossed. Id rather build new links than take down old ones or try to dilute.

    What do you think about using =GoogleIndexCount instead of the Pagerank formula when determining is a backlink is indexed or not? It seemed a bit more accurate but i'm still checking through my list. Pagerank gave me many -1 that were URLs of indexed pages. Is there a downside to the googleindexcount formula that you know of?

    Again - thanks for a great post! Ive sent it along to all my SEOs :)

    Amber

    reply >
  33. I think that after almost three months, most still do not know how to recover, the success stories are minorities, and can not be applied in many cases. Must continue to wait.

    reply >
  34. Sivagopi

    Hi suppose say my site is http://www.example.com. There are around 5000 links that containing the anchor text "example".

    And the same site is affected by penguin.

    Do you think this is one of the case for the hit.

    reply >
    • John Doherty

      Hi Sivagopi - that would not surprise me, actually, especially if the site is not really a brand. If your site is boots.com, people would naturally link with "boots.com" and "www.boots.com" than "boots". If you have a ton of these one-word links that match your domain, it's not going to look natural especially with a money keyword like that. A brand, such as SEOmoz that created their own search volume, will be treated differently, I think.

      Hope this helps.

  35. John,

    First of all, great article. Secondly, now that we are 6 weeks out from your post, are they any key signs that show you were effected by panda?

    I am in the process of diagnosing some panda effected sites and have found this information very useful. Would you say anchor text variation is a pretty key point with panda? Are they looking for brand signals now more than anything?

    Thanks

    reply >
  36. John, you did an amazing work. I would like you to keep us posted on your new founding.
    One of my client who ranked in top3 with a very competitive search terms for 7+ years was trashed by Penguin update. I have not been worrying about Google updates before penguin. I always supported Google updates thinking they were eventually eliminate the cheaters. I was so confident with our work.
    After this Penguin update, I see bunch of sites clearly cheating placed on the first page in my industry. One site(same domain name) was covering 4 Spots on the first page 2 organic results 2 Google place (One fake location). I know they buy links.
    Another site also on the first page just buying links. Their linkbacks coming from unrelated sites. Those sites are clearly showing how you can buy links and how much u need to pay.

    Thanks

    reply >
  37. Hey John,

    Great post, nice to see all these penguin diagnosis smarts tied together so well.

    We have been helping a few businesses with penguin penalties, same story really, a few have a whole bunch of problems from manual link penalties with a side order of panda and penguin and a few fairly classic penguin, traffic bombed on 24th of April type scenarios.

    I am seeing a lot of really bad link profiles, like ghastly, several years of bought, not earned links, not even lets build some content and then point some links at it, just links, in crappy articles, footers, blogrolls etc and people have been getting away with it.

    A good percentage of the jobs I am picking up now seem to have been running their own little link networks. Lots of spam blogs used to link to each other and the 'money site' (i hate that term) and they seem to have been getting away with, rampantly till this point and now, well, in some cases there is really little to save in the link profile, it's just rotten to the core.

    Do you guys have any point where you just say, well, there is nothing we can do here? Such a percentage of the link profile is rotten that if we remove all the bad, there is just going to be nothing left and it will be a huge, thankless and in many cases, impossible task to fix.

    I remember, being old and all, having my first job managing a site and doing the SEO in 2000-2002 and having a bunch of my own sites where I was doing a bunch of dodgy stuff. When it came to working on my works site I had to read up more on the rules, came across Jill Whelan, Doug Heil and some of the old whitehat guard and despite knowing how to manipulate things played it clean and with content and quality.

    After doing that and seeing slow but steady results for a couple of years, watching the business build, I then witnessed the fallout from the Boston update on Webmaster World and all around the web. People lost their businesses and were up in arms - 'how can google do this?' was ringing out around the web.

    Thing is, we are 11 years on from that point, and small businesses seem doomed to keep on making this same mistake. I have penguin clients that are legitimately good guys, nice folks, and it seems that they truly believed that SEO was just handing over £X a month and letting someone get on with it.

    I am not really sure who is to blame. Google does plenty to educate people, I am sure they could do more, having a few quid in the bank and all but if you want the information, then it is out there.

    I have no sympathy for huge corps who get stung, but little people, who have just been led down the garden path by some

    That said, I have been banging this drum for 10 years myself with my clients and whilst I have lost one or two who insisted that "can't we just do X, till we get to Y and then we will do it properly, honest guv", I have seen many of them fall foul of these new updates and again, in many cases, the link profile is r... [continued below]

    reply >
  38. [continued from above] ...otten and I can only really suggest they burn the house down and start again.

    So, after that rambling diatribe, do you guys have any guidelines for salvaging sites where the link profile is horrific? Is it worth saving anything to try and pull them out of the penalty so they can at least attempt to move forward properly?

    Would love to hear your thoughts.

    All the best
    Marcus

    reply >
  39. Damn, left an unfinished sentence in that somehow, phone rang. :S

    reply >
  40. What a great article. It inspires me to dive into Penguin dialysis even though I'm not a believer in panic-reactions. But thank you for good ideas how to attack the fierce Penguin.

    reply >
  41. you have done excellent analysis
    loved reading your post

    reply >
  42. Has anyone had any luck recovering from penguin? If so what steps did you take to return to the top of the rankings?

    reply >
  43. John, this is a great article!

    There's something that is bothering me though. You mention that "the links that this tells you are deindexed do not need to be worried about". I am not sure about this one.

    If a domain/page shows as deindexed in the public search engine results does it mean that Google is not indexing it at all, at least for analysis reasons? If I was Google then I would still need to know which bad neighbors are linking to whom, and if I would not keep those pages in my private index, how would I know then?

    Plus, if Google wants to identify networks at the same time with punishing those using them, they have to keep this data. In this case, would you still need to clean bad links?

    What do you think?

    reply >
  44. Great post. The question is that neil's excel tool shows a -1 for a page being deindexed but when I search google it is indexed. Have you seen this happen?

    reply >
  45. Jeez that was a good post. I just followed it through now to double check my findings from after Penguin. One of my sites still hasn't recovered after cleaning up the linking profile against 2 main keyword terms that got slapped. So I'm looking around for other things to do to fix the problem. Aaron

    reply >
  46. Thanks John, this is a great post!

    A site I am working for has just been hit by the most recent Penguin update. Is it still best to wait for the next iteration (just wait out the transition rank updates) or is it worth submitting a reiclusion request? And any recommendations as how to best clean up the link profile? I'm finding lost of the spammy links charge a fee to remove a link. I'm sure Google will be interested to hear this!

    reply >
    • John Doherty

      Hi Lucy -

      DO NOT SUBMIT A REINCLUSION REQUEST. This is NOT a manual penalty and thus does a reconsideration request will be a waste of time. If spammy sites charge a fee, then pay it. It's worth getting the link removed, IMO. It also depends how deep your site is with bad links. If you can get them removed, great, but it may be possible to linkbuild out of it by getting solid branded anchors.

  47. Hi John,

    Thanks for the reply. That is what I thought! I reckon wait out the iterations, build solid natural links in the meantime and hope to get as many of the spammy links removed as possible.

    reply >
  48. Pawel

    Hi John,

    Thanks a lot for this article. One of my website has been hit by this update and I am trying to recover by removing links from some directories. The problem is that some of them do not respond to my emails and basically refuse to remove the links. Is there any way to let google know about it so they are not counted any more?

    reply >
  49. Hi John, thanks for this advice. really great that someone bothered to actually go through the procedure step by step.

    Now I have a site that has been hit by 5th Oct Penguin refresh. Most inbound external links are from Pligg type sites and point at home page. Google lists about 200 of them in WMT. Anchors are a good mix of partial match and exact match key terms. So I guess the problem for Google was that there are very few branded terms in the mix.

    My question - if fixing an algo hit website is essentially all about fixing the balance of anchor texts, surely the simplest solution is to go out and get even more links, but with branded terms, which should restore the natural balance without the need to delete any links? What do you think of this idea? I would really appreciated your advice as I dont have access to pligg accounts.

    reply >
  50. Now the Google disavow tool is available, this will save a lot of time (and money) removing bad links.

    reply >
  51. John,

    I absolutely dig this post! Before reading I ran a similar experiment in Analytics, but reversed the dates (past in the first box, present in the second). This will give you all of your previous high traffic keywords in order, and what they are getting now. Just a little easier.

    What are your thoughts on internal linking with Penguin?

    reply >
  52. Thank you John for such a wonderful article. I must say, this is one of the top 10 articles over the internet about penguin. BTW thanks for the pre-designed excel which will be very much helpful for me to diagnose :)

    reply >
  53. Wow thank you for posting such a comprehensive action plan in order to diagnose some of these problems! Your detailed descriptions will really help a lot of people to identify where their problems could be coming from.

    reply >
  54. Nice Tip John :)

    Penguin and Pandas now Playing Game on the Internet, Every webmasters and SEO consultants constantly updating their Knowledge and Website Architecture according to the Google Panda Guidelines. But it's not going to end, because google rolling out new updates every month :)

    reply >
  55. Thanks for this exceptional post and all your hard work here. Saves me tons of work trying to explain to my readers what I barely understand. My readers like doing their own pages and this will help them heaps!

    reply >
  56. Thanks John for this great article. The video on Pivot Tables is very helpful.

    reply >
  57. Awesome stuff John. Can't ever know too much about past and present Google updates. Got to stay up to date and make sure the next one doesn't kill ya!

    Love the article.

    Cheers!

    reply >
  58. Wow. This is the best penguin article I've read! Thank you so much for writing it!!!

    reply >
  59. Great post. 1m all for that a page being deindexed but when I search google it is indexed. Can this happen?

    reply >
  60. Still love reading this John :-) and somewhat surprised at the number of folks still going on about 'submitting a reinclusion request', when in certain cases, it just isn't the issue.
    Thanks again for you work :-)

    reply >
  61. This post is what i was looking for researching on how and why google's new crawl system has affected our many websites but not my main website which is now 7 years old.
    I think, one must do a thorough research on this using John's ideas and bring back their websites thrown down by google.
    Thanks John for this information!

    reply >
  62. I used to have 4,000-4,500 unique visitors a day (mostly from search engines). Nowadays I only got around 2,000 a day, sigh. Not sure the silly Panda or Penguin causes it but traffic has been going downhill (looks like some keyword search ranking dropped)

    Considering I write my own article every day (+ other guest posters), it's a bit unfair in a way

    reply >
    • True.. even i do the same.. posting myself and accepting few guest posting articles (only quality ones) but still some dummy sites with one page ranks on 1st page above me! and my site ranks... NOWHERE :(

  63. Really nice article.. I have few sites which got penalized from penguin and till date i am trying to recover it using different analytical methods.. got 10% success too but my competitor blasted my homepage URL using spammy blog comments which makes my job tough :( already submitted all those backlinks (spammy ones) in Google's disavow tool and still waiting for Google to remove that link penalty happened by blog comments.

    Also, i will surely try your diagnosis on it to get few more issues out. Thanks again.

    reply >
  64. Great analysis. I have also been asked this on a daily basis and use a different strategy and your looks good as well. I can share mine as well with you just contact me via skype seohop. Talk to you soon

    reply >
  65. Jason

    I re-read your article, it's great. Thank you for the links to the various xls.
    I would love to get your opinion on whether or not this site was "hit". It is a potential prospect for us and I gave them my opinion of the site, which I don't think they wanted to hear. I would love to get another opinion(s).
    Google Organic traffic, before/after 4/24/2012 http://www.screencast.com/t/ThQ13Vgi
    This second screenshot compares the backlinks to the lost keywords for the domain.

    Thanks!

    reply >
  66. Jason

    I re-read your article, it's great. Thank you for the links to the various xls.
    I would love to get your opinion on whether or not this site was "hit". It is a potential prospect for us and I gave them my opinion of the site, which I don't think they wanted to hear. I would love to get another opinion(s).
    Google Organic traffic, before/after 4/24/2012 http://www.screencast.com/t/ThQ13Vgi
    This second screenshot compares the backlinks to the lost keywords for the domain.
    http://www.screencast.com/t/DerU771g5LrD
    Thanks!

    reply >
    • John Doherty

      Jason - it was hit by something, but I'm not sure what. The drop doesn't correlate with the dates of Penguin 1, that's for sure.

    • John Doherty

      Also, if you want to see what might have happened, you need to at least go back a few weeks before Penguin. Your screenshot doesn't show a before and after of the Penguin update, just before.

  67. Interesting read. Thanks. I have a client who has had some seriously heavy link building from low grade directories over many years with hundreds of exact or similar anchor text. He experienced a massive drop, by around 50% last April and it's bizarre, it seems to be the home page specifically as other urls on the site have been doing very well. The home page keywords seem to recover for a few weeks or even days, then they drop again to out of the top one hundred page.. truly bizarre behaviour, we've tried manually removing links which was proving painfully slow and almost not worth the effort, so we submitted a disavow request. A few weeks later, the site is certainly acting more stable although at a slightly lower level of rank for the affected keywords, which fill sme with a little more confidence and would make sense if disavow has indeed worked, the artificial page rank will now be gone with a bit of luck. There's so much confusion around all this.. so nice to read something concise and well informed.

    reply >
  68. Hi John,

    Great article!

    I carried out a similar analysis on my website (www.maidenheadtown.org) using a combination of the links from Webmaster Tools, Ahrefs and OpenSiteExplorer to give a more rounded overview of my link profile. I then used a certain "box named" (I'm sure you know the program I mean) piece of software to check Google indexation and anchor text, in addition to whether the links are actually still live. So as you can imagine I've been pretty thorough, and have cleaned up my anchor text so that it has a natural distribution. I have also been through the homepage and reduced the keyword density I know Matt Cutts said not to, but I believe I was a bit "over-optimised". Anyway despite all of this my homepage still does not rank for it's main keyword ("maidenhead"), not even in the first 200 results despite having a solid link profile. Have you any idea why this might be? It does however rank no. 1 for "maidenhead town".

    I am certain that it is a Penguin issue as my traffic dropped on the 24th April 2012 and it no longer ranked for this keyword. I also know that it wasn't a manual action as I didn't receive a message through Webmaster Tools.

    Regards,
    Peter

    reply >
  69. The simple fact is to get around a penguin penalty you need to de optimize your anchor text and most importantly, no commercial intent anchor text.

    reply >
  70. If anybody is interested in using the tools mentioned above, in particular the Link Detox Genesis tool, at a reduced rate, I can help. I am putting a community together (the link should be my name in this comment). The idea is to get 10 people together and I will help them get their penalties lifted. Why 10? This is because LRT are bringing out a new tool next week which will help speed up how quickly Google takes notice of the disavow file. Unfortunately the new tool is only available with the higher level LRT plans, which are quite expensive, so my plan is to split the cost 10 ways. In return I will help 10 people get their penalties lifted. I have completed the Link Research Tools Associate Training, but I want to get more experience and some testimonials, so this is the best way I can think of doing it. Membership is free, but limited to 100 before the community becomes a private mastermind.

    Thanks Jayson, that's good advice there. After having my business destroyed by Penguin 1.0 in April 2012, I have been trying to get my manual penalty lifted for a long time. The final bit of success came this month when I was using the Link Detox Genesis tool by Link Research Tools. Using this on it's own wasn't enough so I did the LRT Associate training too. So I have now decided to turn my disaster into a positive and become the 'link clean up guy'! So...

    If anybody is interested in using the tools mentioned above, in particular the Link Detox Genesis tool, at a reduced rate, I can help. I am putting a community together (the link should be my name in this comment). The idea is to get 10 people together and I will help them get their penalties lifted.

    Why 10? This is because LRT are bringing out a new tool next week which will help speed up how quickly Google takes notice of the disavow file. Unfortunately the new tool is only available with the higher level LRT plans, which are quite expensive, so my plan is to split the cost 10 ways. In return I will help 10 people get their penalties lifted.

    I have completed the Link Research Tools Associate Training, but I want to get more experience and some testimonials, so this is the best way I can think of doing it. Membership is free, but limited to 100 before the community becomes a private mastermind.

    reply >
  71. I find it pretty amazing that there are over 450+ negative comments on the official Google webmaster blog about the new algorithm changes and no response from Google or the spam team.

    reply >

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>