Earlier quoted context omitted.
> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…
The real life equivalent of this is "if I leave my door unlocked, should someone be allowed to walk in anyways?" I would definitely want some intent provisions in, but saying something is accessible therefore free game seems too wide.
The problem with analogies is that many equally valid analogies that can be made, but with many different points. I would argue that the real life equivalent is "Have this free book, but you may not read Chapter 4."