Unfortunately rewrite rules on the site meant that
was a valid URL, so returned the appropriate html.
So nice little looping going on there consuming nearly 2 gig in the last 2 days.
According to the crawler's info page:
itsapic.com crawler is designed to respect the robots.txt exclusion directives and META robots tags, and collect material at a measured, adaptive pace unlikely to disrupt normal website activity.I don't call hammering a site day and night "measured, adaptive pace" somehow.
Welcome to the IP ban-list.
Other related posts:
Xero vs. Quickbooks, from a Quickbooks User
Vodafone Website Failure Fails
CSS namespacing, somebody tell me what I'm doing wrong.
Add a comment
Please note: comments that are inappropriate or promotional in nature will be deleted.
E-mail addresses are not displayed, but you must enter a valid e-mail address to confirm your comments.
Are you a registered Geekzone user? Login to have the fields below automatically filled in for you and to enable links in comments. If you have (or qualify to have) a Geekzone Blog then your comment will be automatically confirmed and placed in the moderation queue for the blog owner's approval.