Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Scraping Is Vital to Democracy

themarkup.org

Post reply on HN
Print view View original

121–130 of 132 posts

Re: Web Scraping Is Vital to Democracy

#121

Post by filoleg »

travisporter wrote: post #100
bobobob420 wrote: post #76

Earlier quoted context omitted.

Can the case be made that recording in public is a right (as if always should be) but trying to track where everyone is at every point of time is stalking at a mass scale which should be illegal as stalking on a one to one scale is illegal? For reference this is what one site (findlaw) has given as what constitutes the crime of stalking: The crime of stalking can be simply described as the unwanted pursuit of another…

This sentiment exactly. Sure the information is free. But to find my property records, phone number, and "aggregate them" like Spokeo, fastpeoplefinder and similar sites, is akin to digital stalking IMO

Maybe I am wrong and my take is too simplistic/naive (as I admit, I haven't thought too much about this topic), but imo aggregating publicly available data shouldn't count as stalking. Acting on that data in a way that could be constituted as "stalking" definitely is real stalking though and should count as a crime.

For example, if someone's ex (who was explicitly warned that he was unwelcome) continues showing up at the doorsteps because they remembered your address from a long time ago, then it is stalking. Them knowing the address isn't.

In light of this, I don't see how it matters whether they remembered the address from past experiences or just found it through a website that aggregates publicly available info. As long as the data was obtained legally and without breaking any other harassment clauses, why would just the knowledge of something be a crime?

I see it just like firearms. Having a firearm (in a lot of US jurisdictions) is not a crime, as long as it was obtained legally. Doing harmful actions with it (such as threatening people or shooting someone who didn't pose a threat to your life) is a crime. Having a firearm feels like just having data, in this scenario. As long as you don't use that data for criminal actions, why would just the potential of you being able to do something criminal with that data is a crime?

Re: Web Scraping Is Vital to Democracy

#122

Post by kingdomcome50 »

PestoDiRucola wrote: post #74
notahaterbut wrote: post #69

Web-scraping can do harm to a business if the scraper does not implement things like timeouts or sleeps between processes. Websites and businesses do not factor bandwidth for the rogue scraper.

You can easily block a harmful scraper like that from the website owner's side.

You make the target. I’ll make the bot. Let’s see who wins.

In all seriousness, your suggestion is woefully naive in the face of any serious entity.

Re: Web Scraping Is Vital to Democracy

#123

Post by [deleted] »

Wowfunhappy wrote: post #62
mnmkng wrote: post #20

I think of web scraping as nothing less than automation of human work. There really is no hacking or unauthorized access involved. It's either me, using a browser, to read something publicly accessible on the internet, or it's my computer, which I've programmed to read those things for me, so I can focus on creative work or spend time with friends and family. This applies not only to personal life, but businesses as…

> I think of web scraping as nothing less than automation of human work. I agree that scraping should be allowed. We wouldn't have Google otherwise. However, there's something to be said for the fact that some activities are okay at a small scale but become problematic at a large scale—particularly the scale which becomes possible when a task is automated. For instance, I recently bought an expensive camera and have…

[deleted]

Re: Web Scraping Is Vital to Democracy

#124

Post by colejohnson66 »

josephorjoe wrote: post #116
amelius wrote: post #109

Earlier quoted context omitted.

> The crime of stalking can be simply described as the unwanted pursuit of another person. Examples of this type of behavior includes following a person, appearing at a person's home or place of business, making harassing phone calls, leaving written messages or objects, or vandalizing a person's property. Except for the last point, that sounds like ads :)

ads vandalize property by causing unwanted images to appear on and unwanted sounds to emit from personal property

Is a highway billboard vandalism when the owner of that billboard is allowing the ad?

Or are you referencing computers with ads in them (websites, etc.)? By that definition, anything disproving a conspiracy theorist is vandalism to them. That’s not how that works.

Re: Web Scraping Is Vital to Democracy

#125

Post by narag »

fakedang wrote: post #113
narag wrote: post #61

Earlier quoted context omitted.

If, for example, I start scraping all the articles on a newspaper’s web site, publish them on a web site, adding my own ads, most people would think that shouldn’t be legal. Publishing? Ads?

Republishing, duh.

OK, let's reformulate that: would people be opposed to just scrapping or to the illegal republishing and adding ads?

Another similar trick: "If someone will block ads and then murder the publicist and burn his house, most people would find that it should be illegal".

Re: Web Scraping Is Vital to Democracy

#126

Post by err4nt »

amelius wrote: post #7
1vuio0pswjnm7 wrote: post #6

"In both of those instances, the pages and data scraped are publicly available on the internet-no hacking necessary-but sites involved could easily change the fine print on their terms of service to label the aggregation of that information "unauthorized."" They could. But if they filed a claim under the CFAA without ever sending a cease and desist letter to the alleged intruder, I think the claim would be dismissed.…

So robots.txt has no meaning?

It's not binding, but it's a hint they may listen to where you can say things like if there's a certain URL structure where your server would keep serving content infinitely and just waste the crawlers time and resources, you can hint to them they maybe they should avoid certain URLs, but hey if they wanna waste their resources crawling that anyway be my guest!

Re: Web Scraping Is Vital to Democracy

#127

Post by fakedang »

narag wrote: post #125
fakedang wrote: post #113

Earlier quoted context omitted.

Republishing, duh.

OK, let's reformulate that: would people be opposed to just scrapping or to the illegal republishing and adding ads? Another similar trick: "If someone will block ads and then murder the publicist and burn his house, most people would find that it should be illegal".

Advertising isn't some zero-sum game that suddenly makes you lose because you happened to see it. Republishing is - the original publisher loses out on the value of his publication because that means either reduced ad revenue or reduced subscription. I'm guilty myself of reading republished articles, but it is a loss for the publisher.

Re: Web Scraping Is Vital to Democracy

#128

Post by narag »

fakedang wrote: post #127
narag wrote: post #125

Earlier quoted context omitted.

OK, let's reformulate that: would people be opposed to just scrapping or to the illegal republishing and adding ads? Another similar trick: "If someone will block ads and then murder the publicist and burn his house, most people would find that it should be illegal".

Advertising isn't some zero-sum game that suddenly makes you lose because you happened to see it. Republishing is - the original publisher loses out on the value of his publication because that means either reduced ad revenue or reduced subscription. I'm guilty myself of reading republished articles, but it is a loss for the publisher.

I hope you realize that what you wrote isn't related to my point at all. It's fine, but why put that as an answer to my comment is beyond me.

Re: Web Scraping Is Vital to Democracy

#129

Post by 1vuio0pswjnm7 »

IfOnlyYouKnew wrote: post #13
1vuio0pswjnm7 wrote: post #6

"In both of those instances, the pages and data scraped are publicly available on the internet-no hacking necessary-but sites involved could easily change the fine print on their terms of service to label the aggregation of that information "unauthorized."" They could. But if they filed a claim under the CFAA without ever sending a cease and desist letter to the alleged intruder, I think the claim would be dismissed.…

As to precedent, there is a decision by the 11th circuit asserting such an expansive reading of the law. It's the very decision that's the subject of this appeal. Other than that, you're mixing civil and criminal law rather liberally. I agree that it would be insane to create criminal liability for run-of-the-mill violations of ToS, and there is a decision from the MySpace era saying as much (and not even involving a…

We'll have to wait and see but I believe there should be a relevant distinction between a publicly accessible database like a backend to a public non-governmental website versus a non-public database being served by a protected governemnt computer, such as the ones Van Buren (2019) or Rodriguez (2010) accessed. Not to mention there was an existing (employer-employee) relationship between either Van Buren or Rodriguez and the operator of the computer database: the US government. Compare that to scraping a public website; the person doing he scraping may have relationship with the operator of the website.

I could set "Terms of Use" for the data stored on protected computers I own. I could place limits on "acceptable use" of this data. But can I really argue that I gave adequate notice to all the tech companies and partners that try to access this data.

Re: Web Scraping Is Vital to Democracy

#130

Post by vsl »

Wowfunhappy wrote: post #106
dehrmann wrote: post #103

Earlier quoted context omitted.

The case I thought of is how public records are great, but the way Spokeo and friends scrape them, post them, and sell them isn't.

Right! Ensuring a process has the right amount of friction can actually be super important, to ensure it's available but isn't used wantonly.

This is a great way to phrase it. Friction really is essential for so many things where we're gradually loosing it.
Post reply on HN