I’m a big fan of JS bookmarklets for hacking productivity and automating routine tasks that I do over and over again. In this post I want to share with you a new one that I found and have been using a lot…
If you don’t know what a JS bookmarklet is or how to use them I strongly suggest you click here to get the basics.
Check if a page is blocked by robots.txt
This all came about because of a Google+ post I made:
While doing site audits I regularly find myself diagnosing indexation issues for a particular page and I find it very handy to be able to click a button and see if it’s blocked by the site’s robots.txt file as that’s a common issue.
Thankfully the lovely Jeff Bail came to my rescue and coded me up a little bookmarklet to do this analysis.
Click here to grab the JS bookmarklet
If you add this to your bookmarks, browse to a page that’s blocked by robots.txt and then use the bookmarklet you should see something like this:
Since Jeff sent this over to me last week I’ve been using this like crazy so massive thanks to Jeff. If you want to show him some love tweet him!
I know this is a short post but I wanted to get this out there, thank Jeff and let you guys have a play with this. It might still have a few kinks to iron out so let me or Jeff know if you spot anything not working.
Tom Critchlow Tom Critchlow is VP Operations for the NYC office, living in Brooklyn and working in Manhattan. Fiercely curious about most things and passionate about everything.
Thank you for sharing, this will come in handy.
The “seo” command in my Quix bookmarklet does that and more: it actually casts an “indexation verdict” for Googlebot, checking the robots.txt file, robots meta tags and x-robots-tag http headers.
If you run a check for this page, for instance, you’d see this page can be indexed, but also that the title is too long, at 74 chars
Nice one Tom. I also really like Quix, thanks Joost for the heads up.
Thanks for this. Also hadn’t heard of Quix before, thanks for sharing!
Just found this today after browsing around that new SEOmoz post. I’ve never messed around with bookmarklets but after today, I will be looking more frequently for new ones. This is great though, good stuff!
Hey guys,
Thanks for the comments – there’s a few kinks with the JS that have now been ironed out. Hope it’s useful for you. You might also be interested in this posted today:
http://moz.com/ugc/a-onestop-seo-bookmarklet-to-quickly-review-onsite-seo
@Joost – thanks for dropping the qix knowledge, I’ve tried to get into using that before and never quite got comfortable enough with it to use it regularly but maybe this will be the push I need!