Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accessing Publicly Available Information on the Internet Is Not a Crime

eff.org

Post reply on HN
Print view View original

11–20 of 299 posts

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#11

Post by PatientTrader »

FilterSweep wrote: post #6
PatientTrader wrote: post #5

Earlier quoted context omitted.

> I'd also note that these companies are barely (if ever) held liable for life-compromising hacks on their platforms. You do know it is impossible to stop all cyber attacks? Its always a matter of when, not if. Zero day attacks are developed everyday with not even the best funded cyber security systems able to thwart them. The geniuses are on the offensive side, if they want in, they will get in.

You didn't really address the point you quoted. The problem isn't that someone is getting IN ; it's that the company throws up their hands and says "tough sht." Or in a worse case, when Equifax puts up a compromised site to find if you were hacked that requires a significant amount of your SSN and personal details. (edit: format)

> it's that the company throws up their hands and says "tough sht."

What exactly is your solution to the problem? You are more or less complaining without providing any insights into addressing the issue or without knowledge of the threat landscape.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#12

Post by diggan »

ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

> robots.txt violations being considered hacking

Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa.

It's not about if it's difficult to build but rather the principle behind if you can just allow humans to read something.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#13

Post by anilgulecha »

ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

robots.txt have no legal validity.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#14

Post by ikeboy »

diggan wrote: post #12
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…

Why is discriminating against robots unfair? There are valid reasons (for instance, robots take a lot of resources to serve and don't lead to revenue).

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#15

Post by toomuchtodo »

PatientTrader wrote: post #11
FilterSweep wrote: post #6

Earlier quoted context omitted.

You didn't really address the point you quoted. The problem isn't that someone is getting IN ; it's that the company throws up their hands and says "tough sht." Or in a worse case, when Equifax puts up a compromised site to find if you were hacked that requires a significant amount of your SSN and personal details. (edit: format)

> it's that the company throws up their hands and says "tough sht." What exactly is your solution to the problem? You are more or less complaining without providing any insights into addressing the issue or without knowledge of the threat landscape.

Spending money on security architecture/engineering/pen testing/etc in concert with government regulation/oversight.

Full disclosure: I work in security architecture/risk management in the financial services industry.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#16

Post by rhema »

ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

I don't know. The pathological case could include a rapidly changing robots.txt. Think about archive.org's policy. If they suddenly find new restrictions on a domain, they hide it in their waybackmachine. Sometimes an old site will go down and be replaced by totally new owners. This breaks some domains of the waybackmachine retroactively.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#17

Post by martin-adams »

ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

For the most part I agree, but I feel there are grey areas. Things like web browsers (which are not robots) can access the content as though they are from a human. But what about extensions or apps that do things in the background, such as caching the contents of several pages for offline viewing. Is that now considered a bot.

The robotstxt.org site states that a robot "should" obey the rules. "should" is not a legal term that implies compliance. "must" would have been more appropriate to indicate enforcement.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#18

Post by ikeboy »

anilgulecha wrote: post #13
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

robots.txt have no legal validity.

I mean, it seems to have been cited in the lawsuit. See e.g. https://static1.squarespace.com/static/5803b57737c581885cbd0... and search for it.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#19

Post by ikeboy »

rhema wrote: post #16
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

I don't know. The pathological case could include a rapidly changing robots.txt. Think about archive.org's policy. If they suddenly find new restrictions on a domain, they hide it in their waybackmachine. Sometimes an old site will go down and be replaced by totally new owners. This breaks some domains of the waybackmachine retroactively.

I think judges are able to ask some questions and tell the difference between an honest mistake and a flagrant disregard for robots.txt, if that were to be the legal standard.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#20

Post by diggan »

ikeboy wrote: post #14
diggan wrote: post #12

Earlier quoted context omitted.

> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…

Why is discriminating against robots unfair? There are valid reasons (for instance, robots take a lot of resources to serve and don't lead to revenue).

Just because it's a robot doesn't mean that it takes more resources to load a page. A robot that loads 1000x more pages than a normal user, sure. But then rate-limit everyone rather than blocking specifically bots.

And that bots don't lead to revenue depends on why the bot is navigating on your page no? If it's some indexer that links back to your website and it's a popular index, then you'll maybe end up with more revenue thanks to that bot than a normal user.

Post reply on HN