Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accessing Publicly Available Information on the Internet Is Not a Crime

eff.org

Post reply on HN
Print view View original

281–290 of 299 posts

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#281

Post by grkvlt »

Chickenosaurus wrote: post #184
gwbas1c wrote: post #168

Earlier quoted context omitted.

> Access that does not require authentication should never be a crime. Careful, this could legitimize things like accidental denial of service. Depending on circumstances, even basic scraping could cause problems. (I need to be vague to avoid violating an NDA.) A major internet site had a URL that went something like somedomain/group?id=xxxxx. It turns out that a simple scraper, that called id=1, id=2, id=3, ect, ect…

Accidental denials of service are indeed a common occurrence. By the way, it's "etc" from latin et cetera - I assume you didn't want to refer to electro-convulsive therapy :-)

As any fule kno, this is how Molesworth writes, ect ect ect.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#282

Post by grkvlt »

ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

Interesting. They say that crawling is prohibited there, actually, and have a blanked 'Disallow' at the end.

    # Notice: The use of robots or other automated means to access LinkedIn without
    # the express permission of LinkedIn is strictly prohibited.
    ...
    User-agent: *
    Disallow: /
All the listed bots are only able to access a small subset of pages, the same for each bot apart from one. The 'deepcrawl' bot is privileged, and gets to see the '/profinder' pages, for some reason?

    # Profinder only for deepcrawl
    Allow: /profinder*
Anyone know who operates this bot?

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#283

Post by grkvlt »

jstarfish wrote: post #211
ikeboy wrote: post #159

Earlier quoted context omitted.

Except race /skin color is a legally protected class, and robots aren't (and why should they be? They can't enter into contracts, conduct business, etc. So it's perfectly legitimate to exclude them from a site where they cannot use it in the intended manner). "If you truly didn't want trespassers you should've put up a gate."

Bots being legally protected as a class or not, using robots.txt as the ultimate test of what distinguishes normal traffic from CFAA violations is a very flawed mechanism. It turns your website into a minefield. As a property owner, a no-trespassing sign won't protect you from the lawsuits that result when a toddler drowns in your pool. You're expected to do more (like putting up that gate). Equifax's systems are pep…

> won't protect you from the lawsuits that result when a toddler drowns in your pool

Good analogy. I wonder i operating fa poorly secured website that leaks private information could be seen as an 'Attractive Nuisance' [0] and the owners could be prosecuted for that, rather than the hackers!

0. https://en.wikipedia.org/wiki/Attractive_nuisance_doctrine

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#284

Post by [deleted] »

UncleEntity wrote: post #25

How does a website put reasonable limits on access? I'm not saying what Linkedin is trying to do is right but it seems to me there needs to be a way to say "Dude, that's not cool." A regular B&M store can refuse service to disruptive people and trespass people who don't comply, why not servers? --edit-- Pretty much what rayiner is saying, they posted while I was typing.

[deleted]

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#285

Post by harpiaharpyja »

carterehsmith wrote: post #257
odorousrex wrote: post #181

Earlier quoted context omitted.

>A major internet site had a URL that went something like somedomain/group?id=xxxxx. It turns out that a simple scraper, that called id=1, id=2, id=3, ect, ect, caused a major problem! This is a failure on the part of the developers at that "major internet site". Using a guid instead of consecutive IDs, a rate limiter, hell even just a cache...or all of the above. There are lots of solutions here. You have to take ro…

>> This is a failure on the part of the developers at that "major internet site". Using a guid instead of consecutive IDs, a rate limiter, hell even just a cache...or all of the above. There are lots of solutions here. You are right, but few organizations are sophisticated.. or wealthy enough to employ all of that. I mean, a couple years ago there was a thing that Google's Docs could be enumerated. And that's Google,…

Fair enough. It still shouldn't become a criminal issue.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#286

Post by TheCoelacanth »

decasteve wrote: post #252
ikeboy wrote: post #9

>good bots You mean, bots that obey robots.txt? https://www.linkedin.com/robots.txt very specifically prohibits scraping by any bot besides a small whitelist. robots.txt compliance is not difficult to build. I'm fine with robots.txt violations being considered hacking.

Do Not Track compliance is even easier to build. Does the same logic apply?

Yes, this is an extremely good point. If failing to follow robots.txt is a criminal violation of CFAA, then using any of my computers resources (cookies, javascript, etc) to track me while I am sending a DNT header is also a criminal violation of CFAA.

I would almost be willing to concede making not following robots.txt a violation of CFAA if the trade-off was Mark Zuckerberg being brought up on several billion felony charges every year.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#287

Post by bhhaskin »

jacquesm wrote: post #237
us0r wrote: post #193

What makes this extra ridiculous is the fact LinkedIn built its business on scraping not publicly available information but private address books of unsuspecting users.

And spamming those contacts with requests to join that looked as if they originated from your business relations when that definitely wasn't the case.

It's the main reason I don't have a LinkedIn and never will. They are a scummy company.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#288

Post by someonewithpc »

feelin_googley wrote: post #203

Here is an example of the "good bot"/"bad bot" nonsense in action. This is an article about the LinkedIn v hiQ case at AdWeek. curl --user-agent INSERT_ANYTHING_HERE http://www.adweek.com/digital/rami-essaid-distil-networks-guest-post-linkedin-hiq-labs/ It seems AdWeek can distinguish a "good bot" from a "bad bot" irrespective of the behavior of the user^W bot, i.e., whether it is one single HTTP request or 10,000 co…

A related story is that windows 9 isn't a thing because software used to check for windows 95 and 98 by matching the name to "windows 9".

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#289

Post by misterhtmlcss »

Can data that is supplied with an intention to be publicly accessible i.e. public domain be restricted. If the public was asked, "When you supplied your picture, your name, and then created a public URL to become fully searchable, was your intention that that information was to be restricted or was your intention that this was information you publicized about yourself to make it possible for potential employers to find you?". Answer, "Yes, it was 100% my intention to become searchable so that employers would be able to seek me out". Conversation is over.

LinkedIn creates an implied covenant with public consent (mostly) to then publish and make discoverable their professional profiles.

While LinkedIn 100% should have the right to stop others from embedding without permission since it's possible to claim the data structure and presentation is proprietary to them, this should never extend to the actual data itself, since this was willing gifted by the actual owners (Joe public) into public domain.

I think an argument could be made that LinkedIn is being burdened with a degree of data mining that affects their business and therefore should be able to charge a minimal fee e.g. an API firehose to acquire the data in bulk from providers in an raw data stream.

That seems reasonable depending on the charges associated with that offer, this would be the correct compromise, since their data structure is all that actually separates their service from say About.me or any other site of that type. All of which don't disallow scraping; as long as it doesn't present as a DOS attack (of course).

Anyway my comments are as a marketer and not a programmer or lawyer, but personally I'm very interested to see this case resolved in a manner that doesn't suit LinkedIn in the slightest.

Re: Accessing Publicly Available Information on the Internet Is Not a Crime

#290

Post by nathan_long »

gkya wrote: post #164
nathan_long wrote: post #120

Earlier quoted context omitted.

> It should be ensured that certain practices are followed in security Let's not legislate specific practices. Imagine if we had security legislation from 1995 to follow when programming today. Imagine trying to explain to senators why last year's XSS protection rules need updating. Imagine Oracle lobbying to get their database enshrined as the "security-compliant" one. The law should focus on outcomes: if a site get…

"Security compliance" is about how you use a given database, not which one you happen to use. You can securely (but inefficiently) store credentials in a plain text file. WRT some defences becoming outdated by time, well, it probably would not be two-decades behind, but a couple years or so at most. Even then, ensuring that is better then nothing. People need tools to judge if they can safely use some product, and th…

> WRT some defences becoming outdated by time, well, it probably would not be two-decades behind, but a couple years or so at most. Even then, ensuring that is better then nothing.

Not necessarily. What if the law mandates use of, say, an encryption algorithm that has been cracked? You can't move to a new one without breaking the law.

Post reply on HN