Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accessing Publicly Available Information on the Internet Is Not a Crime

eff.org

Post reply on HN
Print view View original

241–250 of 299 posts

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#241

Post by JepZ »

gwbas1c wrote: post #168
ynniv wrote: post #60

Access that does not require authentication should never be a crime. If LinkedIn wants the courts to intervene, they must require authentication for their data. If they also want Google to scrape their site, they must require Googlebot to authenticate itself.

> Access that does not require authentication should never be a crime. Careful, this could legitimize things like accidental denial of service. Depending on circumstances, even basic scraping could cause problems. (I need to be vague to avoid violating an NDA.) A major internet site had a URL that went something like somedomain/group?id=xxxxx. It turns out that a simple scraper, that called id=1, id=2, id=3, ect, ect…

Honestly, we all know the wild west mentality of the internet (yes, it is post national as in 'above the law') and therefore, everyone should assume attacks like that and build defenses against them. Building a service which could be brought to a 'major problem' with simple requests leading to high server loads is just negligent. What would that site do if someone actually wanted to attack it?!?

I am not saying that the trouble with the law enforcement in the internet is a neither a good nor a bad thing. Actually, it depends and in the 'real' world I am pretty happy that the law enforcement works quite good where I live. I think the thin line is somewhere where I start to fear my own governments more than the bad guys (while not having any evil intentions or plans at all).

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#242

Post by bottled_poe »

jstarfish wrote: post #218
dec0dedab0de wrote: post #206

Earlier quoted context omitted.

No, that is a failure of the developer of the scraper. I am definitely pro scraping, but you have to be a good neighbor.

How the hell is the scraper dev supposed to anticipate how poorly-written these particular views are with no backend knowledge? If not an automated scraper, a thundering herd from content gone viral would trigger the same result.

Scraping is not an intended purpose for most websites. Unless the website specifically states that this is an intended function, it is not reasonable to assume so. In fact it may be in violation of the terms and conditions of the given website.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#243

Post by rtpg »

diggan wrote: post #12
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…

The real life equivalent of this is "if I leave my door unlocked, should someone be allowed to walk in anyways?"

I would definitely want some intent provisions in, but saying something is accessible therefore free game seems too wide.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#244

Post by JepZ »

guywaffle wrote: post #195

Make it fair then! Bots can’t scrape LinkedIn, and LinkedIn can’t sell any consumer data to third parties.

For real: I really hate corporations 'stealing' data from my phone. For example Google likes to introduce new sync options to Android and every time they do so it is activated by default. So as soon as the update arrives their software syncs my data to their servers without my consent. They probably have some clause in the EULA but as a user of their products I really hate that behavior. A similar case is not being able to disable address book sync before it syncs the for the first time.

Those things should be crimes as the data they fetch is not publicly available on some web page but exists only on my personal device and they take it without my consent.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#245

Post by kevin_thibedeau »

bottled_poe wrote: post #242
jstarfish wrote: post #218

Earlier quoted context omitted.

How the hell is the scraper dev supposed to anticipate how poorly-written these particular views are with no backend knowledge? If not an automated scraper, a thundering herd from content gone viral would trigger the same result.

Scraping is not an intended purpose for most websites. Unless the website specifically states that this is an intended function, it is not reasonable to assume so. In fact it may be in violation of the terms and conditions of the given website.

I don't intend people named Steve to access my open site so I can sue all Steve's for their felonious behavior?

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#246

Post by _mhr_ »

paulus_magnus2 wrote: post #216

How about: if you want me not to scrape it, keep it off my internet?? Actually I'm considering building "API-fication" of websites with bindings for major languages (Java, Python, JS). With luck websites could participate by providing & maintaining a parseable API-sitemap. This would open door to my 2nd project: orchestration a-la BPEL on top of websites. visual editor, macros, scripting. Call this PIPES 2.0

Can you provide some use-cases for why this would be useful in a way that wouldn't violate most sites' ToS?

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#247

Post by nostrademons »

bottled_poe wrote: post #242
jstarfish wrote: post #218

Earlier quoted context omitted.

How the hell is the scraper dev supposed to anticipate how poorly-written these particular views are with no backend knowledge? If not an automated scraper, a thundering herd from content gone viral would trigger the same result.

Scraping is not an intended purpose for most websites. Unless the website specifically states that this is an intended function, it is not reasonable to assume so. In fact it may be in violation of the terms and conditions of the given website.

If the law assumed that only intended functions are permissible, innovation would be a crime. By definition, innovation is finding new and unforeseen uses for resources.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#248

Post by chii »

diggan wrote: post #12
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…

> ACL for accessing the information

the ACL is the robots.txt. A door with or without a lock doesn't determine whether the place is public or not.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#249

Post by joelanford »

chii wrote: post #248
diggan wrote: post #12

Earlier quoted context omitted.

> robots.txt violations being considered hacking Hm, I disagree. Either information is public, no matter for who. Or the information is private, and you should have ACL for accessing the information. I don't think it's fair to say that information is public if you're a human but private if you're a machine, or vice versa. It's not about if it's difficult to build but rather the principle behind if you can just allow…

> ACL for accessing the information the ACL is the robots.txt. A door with or without a lock doesn't determine whether the place is public or not.

robots.txt is more like a sign that asks certain people not to look at a bunch of other publicly visible signs.

One can't post a sign in public that tells people not to look at other publicly visible signs and expect the government to arrest or fine them for ignoring it.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#250

Post by mgalka »

ynniv wrote: post #60

Access that does not require authentication should never be a crime. If LinkedIn wants the courts to intervene, they must require authentication for their data. If they also want Google to scrape their site, they must require Googlebot to authenticate itself.

> Access that does not require authentication should never be a crime.

Linkedin is not trying to prevent access. They want to prevent information from being scraped, and then used to their detriment.

Post reply on HN