Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Congrats! Web scraping is legal! (US precedent)

parsers.me

Post reply on HN
Print view View original

381–390 of 409 posts

Re: Congrats! Web scraping is legal! (US precedent)

#381

Post by beerandt »

dragonwriter wrote: post #309
beerandt wrote: post #209

Earlier quoted context omitted.

Across the land within their circuit, over matters within their jurisdiction. Elsewhere the ruling is merely advisory in nature. It gets tricky with nationwide actors though. Besides some specialized topics like patents and international trade, nationwide orders and injunctions are the sort-of exception, which are based on a courts local jurisdictional power over a non-local nationwide actor. I'm not sure what the ge…

> One of the usual requirements for the Supreme Court to even hear a case is that different circuits have conflicting rulings on a matter. That's not a “usual requirement”, it's one of many factors that can weigh in favor of the Supreme Court exercising discretionary appellate jurisdiction (and it's one that weighs very heavily in favor of it, even when no other favorable factors are present, since federal law meanin…

Well of course that's what I meant by "usual requirement."

How else would one read it? :)

Re: Congrats! Web scraping is legal! (US precedent)

#382

Post by coryfklein »

briandear wrote: post #252
orf wrote: post #237

Earlier quoted context omitted.

Provide an API for public data to reduce the costs associated with rendering a full blown page, and deliver just the information needed.

Who pays for that API and the bandwidth? What’s in it for the data provider? On LinkedIn, viewing the data now shows ads or at least prompts the viewer to join the network. With scrapping and free API access, how exactly does LinkedIn benefit for their work of hosting the data?

Look, if you build a product that relies on providing free information to the public, then you don't get to select a segment of that public and charge them for it. You can't hang a billboard on a highway, but then get upset when some people look at it the wrong way.

Now, if you want to have a walled garden and charge for entry to some and let others in free then that is fine.

Re: Congrats! Web scraping is legal! (US precedent)

#383

Post by brink »

ramblerman wrote: post #27
brink wrote: post #19

Earlier quoted context omitted.

I think you mean contradiction, not paradox.

Not really. Both are correct, and paradox fits better in OPs context, paradox: a statement or proposition that seems self-contradictory or absurd but in reality expresses a possible truth.

A paradox is something that initially looks like a contradiction, but after closer inspection, is not.

Re: Congrats! Web scraping is legal! (US precedent)

#384

Post by sillyquiet »

a1369209993 wrote: post #367
sillyquiet wrote: post #52

Earlier quoted context omitted.

Yeah, I have nothing at all against scraping per se, it's more about the huge bot traffic the commercial scrapers generate, which would ALSO be fine, but 1. it can be hard to tell scrapers from malicious DDOS bots sometimes and 2.) the person being scraped literally pays for that scraping traffic.

So, apply DOS/DDOS mitigations? You need to have those anyway, so you might as well use them for this too.

Yeah, but where do you draw the line between 'oh sorry we dropped your request because of rate limiting' (or whatever mitigation strategy) and 'oh, we dropped your request because u scraping us bro' legally? IANAL, but this lawsuit seems to indicated that putting barriers in front of scraping attempts is a no-no.

Re: Congrats! Web scraping is legal! (US precedent)

#385

Post by mic47 »

buboard wrote: post #374
mic47 wrote: post #343

I strongly disagree that not allowing scraping protection on social network is a good thing (reasons mostly from this thread https://news.ycombinator.com/item?id=22182144 ) What I would do on Linkedin side is: 1. Split public setting into 2 settings: public for everyone including scraping, and second choice that make it public for Linkedin users (aka banning scraping [1]) 2. Everyone on Linkedin who have current sett…

Scrapers can just create a user account then. They still seem to be covered by this injunction

Legal: It will be hard to argue in court that you have right to create fake accounts. That is definitely against TOS, and at that point Linkedin can sue the company instead, so I believe it would scraper would not be successful at court.

Technical: This is actually what you want, as limiting scraping per user is easier than per IP, since IP can be shared by users, so your rate limits have to be higher for IP address than for individual user. Additionally, creating fake account is more work which will in the end making scraping more expensive.

Re: Congrats! Web scraping is legal! (US precedent)

#386

Post by Eli_P »

AznHisoka wrote: post #5

Linkedin is taking this to the Supreme Court: https://www.law360.com/articles/1237505/linkedin-will-go-to-... No ultimate decision was ever made, and no, this doesn't make web scraping 100% legal. Wake me up when there's a new announcement because anyone interested in this already know this old news.

The link you gave, says (sic):

> Sign up now for free access to this content > [...] > Email (NOTE: Free email domains not supported)

I think this sort of antinomy[1] sounds ironical especially in this thread. This is practically legal dependency injection pattern, when you call something free, then administer antinomies, catches-22, etc.

[1] https://en.wikipedia.org/wiki/Antinomy

Re: Congrats! Web scraping is legal! (US precedent)

#387

Post by msla »

kerkeslager wrote: post #378
msla wrote: post #163

Earlier quoted context omitted.

Making organization membership public would trample on personal privacy quite effectively in some respects, such as with disease support groups or PACs; medical privacy is taken seriously, but is there such a thing as political affiliation being private? Is it a violation of someone's privacy to reveal they give to the ACLU?

The entire point of organizational transparency is to prevent organizations from trampling the rights of individuals, so in cases where organizational transparency would trample the rights of individuals, the rights of individuals supersedes the need for organizational transparency.

So, should it have become public knowledge that Brendan Eich donated $1,000 to support Proposition 8?

Re: Congrats! Web scraping is legal! (US precedent)

#388

Post by kerkeslager »

anticensor wrote: post #380
kerkeslager wrote: post #371

Earlier quoted context omitted.

That's how things are , not how they should be .

No rights, full liability would be a bad deal too.

A bad deal for whom?

It's a bad deal for corporations, but I do not care. Lack of liability is the cause of a ton of problems in our society.

Just to pick two stories of corporate sociopathy: Probably the reason people at State Farm are unconcernd about forging signatures[1] is that they know that the worst case scenario is that State Farm loses some business and maybe gets a fine: they are unlikely to go to jail for forgery or to have fines exacted from their personal bank accounts. Similarly, when Practice Fusion literally killed people[2], their execs had little to fear: nobody went to jail, nobody was fined: shareholders who had no visibility into the decision paid the fines.

When banks tanked the economy with irresponsible lending most were bailed out and gave their workers bonuses, while the people who were unable to pay their mortgages were ignored.

A little more liability for destructive behavior would be great for most people.

[1] https://news.ycombinator.com/item?id=22177812

[2] https://news.ycombinator.com/item?id=22165946

Re: Congrats! Web scraping is legal! (US precedent)

#389

Post by kerkeslager »

msla wrote: post #387
kerkeslager wrote: post #378

Earlier quoted context omitted.

The entire point of organizational transparency is to prevent organizations from trampling the rights of individuals, so in cases where organizational transparency would trample the rights of individuals, the rights of individuals supersedes the need for organizational transparency.

So, should it have become public knowledge that Brendan Eich donated $1,000 to support Proposition 8?

I'm gonna need some context there. To begin with you can't just donate to support a bill. What you probably mean is that he donated to an organization or politician who supported Prop 8, so say that.

The important detail is whether the funds were his personal funds, or whether they belonged to his company: i.e. was he acting on his own behalf, or on behalf of an organization?

It's an inane question, though, because the inane point you're trying to make is that individual privacy might protect homophobes. That's true, but it would also protect gays and allies who donated to oppose proposition 8. The fact that privacy allows people to secretly donate to political campaigns is the point. Part of the reason gay marriage is legal today is that people donated to people like Harvey Milk, at a time when donating to the campaign of a gay governor was risking your job and social standing.

Human rights still apply to humans who do bad things. If you are willing to give up human rights to fight bad people trying to do bad things, then those rights won't be there to protect good people trying to do good things, either.

Post reply on HN