I’m a big fan of JS bookmarklets for hacking productivity and automating routine tasks that I do over and over again. In this post I want to share with you a new one that I found and have been using a lot…
If you don’t know what a JS bookmarklet is or how to use them I strongly suggest you click here to get the basics.
Check if a page is blocked by robots.txt
This all came about because of a Google+ post I made:
While doing site audits I regularly find myself diagnosing indexation issues for a particular page and I find it very handy to be able to click a button and see if it’s blocked by the site’s robots.txt file as that’s a common issue.
Thankfully the lovely Jeff Bail came to my rescue and coded me up a little bookmarklet to do this analysis.
If you add this to your bookmarks, browse to a page that’s blocked by robots.txt and then use the bookmarklet you should see something like this:
Since Jeff sent this over to me last week I’ve been using this like crazy so massive thanks to Jeff. If you want to show him some love tweet him!
I know this is a short post but I wanted to get this out there, thank Jeff and let you guys have a play with this. It might still have a few kinks to iron out so let me or Jeff know if you spot anything not working.
Tom Critchlow Tom Critchlow is VP Operations for the NYC office, living in Brooklyn and working in Manhattan. Fiercely curious about most things and passionate about everything.