JS Bookmarklet for Checking If A Page Is Blocked by Robots.txt

I’m a big fan of JS bookmarklets for hacking productivity and automating routine tasks that I do over and over again. In this post I want to share with you a new one that I found and have been using a lot...

If you don’t know what a JS bookmarklet is or how to use them I strongly suggest you click here to get the basics.

Check if a page is blocked by robots.txt

This all came about because of a Google+ post I made:

While doing site audits I regularly find myself diagnosing indexation issues for a particular page and I find it very handy to be able to click a button and see if it’s blocked by the site’s robots.txt file as that’s a common issue.

Thankfully the lovely Jeff Bail came to my rescue and coded me up a little bookmarklet to do this analysis.

Click here to grab the JS bookmarklet

If you add this to your bookmarks, browse to a page that’s blocked by robots.txt and then use the bookmarklet you should see something like this:

Since Jeff sent this over to me last week I’ve been using this like crazy so massive thanks to Jeff. If you want to show him some love tweet him!

I know this is a short post but I wanted to get this out there, thank Jeff and let you guys have a play with this. It might still have a few kinks to iron out so let me or Jeff know if you spot anything not working.