Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Congrats! Web scraping is legal! (US precedent)

parsers.me

Post reply on HN
Print view View original

401–409 of 409 posts

Re: Congrats! Web scraping is legal! (US precedent)

#401

Post by AstralStorm »

andrewmutz wrote: post #301
rohansingh wrote: post #173

Earlier quoted context omitted.

> Are you sure about this? I am not a lawyer, but I believe that the Terms of Service applies to all users, not just those that explicitly set up a user account. How would that even work? If I browse to any random public page of your website, it's served to me before you've even transmitted the terms of service. How could I be bound by those terms of service when I haven't even seen them?

As an engineer, I agree with what you are saying, but I think normal people and the courts disagree. I think these sorts of contracts are called Adhesion Contracts ( https://www.investopedia.com/terms/a/adhesion-contract.asp ) and we interact with them all the time. For example, if you valet your car, the valet will hand you a piece of paper with a number printed on it to retrieve your car. On that paper you will fin…

This does not work at least for software licensing based on precedents for shrink-wrap contracts, so again would not work for licensing use of data.

A paper served you by the valet is not an immediate contract as you can deny agreeing to it and service does not happen.

You cannot do that with a publicly visible website, unless you show ToS and require agreement before first use. If you allow a non-transferable license then said data cannot be used by a search engine. If it's transferable you just pushed the problem towards scraping a different bot. (Well, you could have a direct agreement with a few major search engines.)

Caveat emptor: not a lawyer.

Re: Congrats! Web scraping is legal! (US precedent)

#402

Post by AstralStorm »

turc1656 wrote: post #320
cabaalis wrote: post #244

Earlier quoted context omitted.

> perhaps the owner of the site has a case. Except it sounds like the owner doesn't. If the information is on the page made public, the owner of the page can't place terms on what is done with the data downstream. They'd have to implement some real binding system such as authentication where CFAA would apply. (IANAL)

Correct, but all of that is void if the data presented is any sort of protected information (copyright, IP, etc.). You can't, for example, scrape Yahoo Finance for pricing and dividend history and republish on your own stock tools website. They have a license to redistribute that data and publish on their own website. Similar story for copyrighted text and things of that nature.

That would require at least showing that ToS on first use. A link on a page is insufficient.

And said ToS would have to force copyright reassignment rather than a general licence, making LinkedIn culpable for any unlawful content published by users of its site.

Re: Congrats! Web scraping is legal! (US precedent)

#403

Post by anticensor »

kerkeslager wrote: post #388
anticensor wrote: post #380

Earlier quoted context omitted.

No rights, full liability would be a bad deal too.

A bad deal for whom? It's a bad deal for corporations, but I do not care. Lack of liability is the cause of a ton of problems in our society. Just to pick two stories of corporate sociopathy: Probably the reason people at State Farm are unconcernd about forging signatures[1] is that they know that the worst case scenario is that State Farm loses some business and maybe gets a fine: they are unlikely to go to jail for…

> they are unlikely to go to jail for forgery

Forgery is criminal regardless of private or official document is concerned. Even in a military setting, forgery of business-related documents are illegal.

> A little more liability for destructive behavior would be great for most people.

Why not full rights, full liability? Replace imprisonment and death by temporary and permanent suspension of company (including re-establishment of a sequel organisation out of a subset of stakeholders) respectively; and voila.

Re: Congrats! Web scraping is legal! (US precedent)

#404

Post by kerkeslager »

anticensor wrote: post #403
kerkeslager wrote: post #388

Earlier quoted context omitted.

A bad deal for whom? It's a bad deal for corporations, but I do not care. Lack of liability is the cause of a ton of problems in our society. Just to pick two stories of corporate sociopathy: Probably the reason people at State Farm are unconcernd about forging signatures[1] is that they know that the worst case scenario is that State Farm loses some business and maybe gets a fine: they are unlikely to go to jail for…

> they are unlikely to go to jail for forgery Forgery is criminal regardless of private or official document is concerned. Even in a military setting, forgery of business-related documents are illegal. > A little more liability for destructive behavior would be great for most people. Why not full rights, full liability? Replace imprisonment and death by temporary and permanent suspension of company (including re-esta…

That's an optimistic theory, but read the link I posted and decide for yourself if you think anybody is going to end up in jail for it.

It seems like you're just picking out one aspect of each of my posts to disagree with. Do you have any disagreement with my overall point?

Re: Congrats! Web scraping is legal! (US precedent)

#405

Post by wtetzner »

jakelazaroff wrote: post #268
wtetzner wrote: post #107

Earlier quoted context omitted.

That's the thing. Web scraping isn't really the problem here. It's what companies are doing with personal information. If LinkedIn started doing the same thing as HiQ, it would be just as bad (probably worse), but the legality of web scraping is irrelevant to that.

That's a good point, and we certainly should write our data protection laws to prevent LinkedIn from doing the same thing — but there's a crucial difference between the two. I've consented to give my data to LinkedIn, and I can withdraw my consent and data if they start doing something I don't like. On the other hand, hiQ has vacuumed up my data without my consent, and there's really no way for me to stop them other…

I guess that's the issue. We need laws to control what companies can do with personal information, even if it happens to be publicly available. I don't think scraping itself is really the issue. If you used mechanical turk to hire a bunch of people to go look and user profiles and write down information about them, you'd have the same problem.

Re: Congrats! Web scraping is legal! (US precedent)

#406

Post by msla »

kerkeslager wrote: post #398
msla wrote: post #397

Earlier quoted context omitted.

So a person spends money to support an odious cause, and that person gets money by being associated with an organization. You hate the odious cause, so you want to avoid giving your money to support people who will then give money to promoting that odious cause. How does that work? Do you no longer have the right to not indirectly support things in your ideal system?

Since you're just openly ignoring the post you're "responding to", I'll just copy-paste my response to what you have just said, with some minor changes: > So a person spends money to support an odious cause, and that person gets money by being associated with an organization. You hate the odious cause, so you want to avoid giving your money to support people who will then give money to promoting that odious cause. Ho…

> Since you're just openly ignoring the post you're "responding to", I'll just copy-paste my response to what you have just said, with some minor changes:

I'm attempting to clarify my question by removing irrelevant details, which it looked like you got hung up on last time.

> Human rights still apply to humans who support odious causes. If you are willing to give up human rights to fight bad people trying to support odious causes, then those rights won't be there to protect good people trying to support virtuous causes, either.

So you are willing to make it impossible for people to boycott organizations as a means of social change, as long as those organizations had an arms-length relationship with anything "political".

Re: Congrats! Web scraping is legal! (US precedent)

#407

Post by EForEndeavour »

AgloeDreams wrote: post #88
dsr_ wrote: post #55

Earlier quoted context omitted.

It does, and the result is that phone books and maps get fictional entries inserted in order to prove copying -- because it is perfectly legal to do your own work to amass the same data set. https://en.wikipedia.org/wiki/Fictitious_entry

Like a Paper town or that time that Genius caught Google stealing their lyrics by playing with straight and curly quotes and writing a tool that watermarked songs by interchanging those quotes on the site. It was brilliant.

You might even say it was Genius.

Re: Congrats! Web scraping is legal! (US precedent)

#408

Post by kerkeslager »

msla wrote: post #406
kerkeslager wrote: post #398

Earlier quoted context omitted.

Since you're just openly ignoring the post you're "responding to", I'll just copy-paste my response to what you have just said, with some minor changes: > So a person spends money to support an odious cause, and that person gets money by being associated with an organization. You hate the odious cause, so you want to avoid giving your money to support people who will then give money to promoting that odious cause. Ho…

> Since you're just openly ignoring the post you're "responding to", I'll just copy-paste my response to what you have just said, with some minor changes: I'm attempting to clarify my question by removing irrelevant details, which it looked like you got hung up on last time. > Human rights still apply to humans who support odious causes. If you are willing to give up human rights to fight bad people trying to support…

> So you are willing to make it impossible for people to boycott organizations as a means of social change, as long as those organizations had an arms-length relationship with anything "political".

No. Please try to respond to what I actually say instead of making stuff up; this is a straw man argument.

There are plenty of other ways we could find out about organizations supporting odious causes and boycott those organizations, without violating the privacy of their members. In fact the the point of "organizational transparency" is to make it hard to hide when organizations do bad things.

In addition to accusing me of saying things I didn't say, you're ignoring what I actually did say. Are you willing to make it impossible for people to privately donate to virtuous causes as a means of social change, when donating to support those causes publicly so would be a risk to their careers and reputations? I'm not going to continue this conversation further if you won't respond to this point.

Re: Congrats! Web scraping is legal! (US precedent)

#409

Post by EGreg »

Aperocky wrote: post #187
AznHisoka wrote: post #5

Linkedin is taking this to the Supreme Court: https://www.law360.com/articles/1237505/linkedin-will-go-to-... No ultimate decision was ever made, and no, this doesn't make web scraping 100% legal. Wake me up when there's a new announcement because anyone interested in this already know this old news.

Bad look on LinkedIn, if you want the information to be visible publicly, don’t fault others for learning that information. Gate the information behind a login then sue the scraper for violating TOS and not scraping itself, I can understand that.

Doesnt this ruling say login gate doesnt matter
Post reply on HN