I use web scraping on my employer's Oracle EBS instance because coordinating with the battle-axe personalities that run ERP's is such a pain in the ass I would rather have a root-canal. With Web Scraping, I don't even have to talk to them.
Web Scraping Is Vital to Democracy
41–50 of 132 posts
Re: Web Scraping Is Vital to Democracy
#42Earlier quoted context omitted.
> That's why I understand why it is a problem for those who publish this data. I still don't understand why it's a problem for those who publish the data. If the "data" is facts (e.g. lists of values associated with objects, like the colors available on automobiles), this data is not protected by copyright, there is no remedy if it's republished, and if a business relies on limiting access to this data, that business…
Databases are actually protected under some copyright laws in some countries.
For copying the database wholesale, yes. But the individual records, formatted on a web page, and scraped out by a bot is not copying the database files or its schema wholesale. They're still individual facts.
Re: Web Scraping Is Vital to Democracy
#43I think of web scraping as nothing less than automation of human work. There really is no hacking or unauthorized access involved. It's either me, using a browser, to read something publicly accessible on the internet, or it's my computer, which I've programmed to read those things for me, so I can focus on creative work or spend time with friends and family. This applies not only to personal life, but businesses as…
While I have substantial agreement with your point of view, when web content is substantially ad-funded, automated scraping effectively bypasses the “payment”. I say this as someone who runs an adblocker, installs them for family, and doesn’t derive income from hosting ads, so I’m not pro-ad; I just realize that while it’s not a crime for me to dump the whole bowl of waiting room candies into my backpack, that’s goin…
When you dump the bowl of candies into your backpack, there are no more candies left for anyone else. On the other hand, when you scrape, all the original data and its underlying use case remain just as they are. You simply having scraped took nothing away from anyone.
Re: Web Scraping Is Vital to Democracy
#44Earlier quoted context omitted.
Advertisements are harmful to every layer of society, mostly because they prey on you to instill desires that you probably wouldn't have had on your own (it being the case that this is their entire added value proposition). They should not be tolerated, and the fact that they ever were is a travesty. That being the case, any argument founded on "but advertisements" does not hold water.
It's not founded on "but advertisements" it's founded on "but paying for content". The fact that websites are developed by people (wages), hosted on infrastructure (hardware renting) and require countless other jobs is somehow magically forgotten in these discussions.
Whatever can be killed by adblockers should be. 95% of the web consists of bloated javascript made by overpaid frontend devs that's less functional than a plain text file.
So the ad money runs dry and nobody can afford to push 20 megabytes of javascript and fonts with every page view anymore. What a tragedy.
Re: Web Scraping Is Vital to Democracy
#45This article focuses a lot on _good_ of web-scraping but I don't think we even need to go that far. Fight against web-scrapers just seems like a complete logical oxymoron: they want data to be public but also select who gets to see it. Our whole web infrastructure are based around clearly distinct public/private exchanges - there's no middleground this and yet people create these absurd hacks like captchas and finger…
[deleted]
Re: Web Scraping Is Vital to Democracy
#46If you post a file to the WWW (World Wide Web) do not be shocked that the whole world can read and copy it. That's how it works. You're welcome.
Re: Web Scraping Is Vital to Democracy
#47Okay, great point, but who will pay for that? Serving content costs money: traffic, extra load on CPU and storages, and now on serverless functions too.
One spider may be equivalent of hundreds of real users. And it spoils caches too.
Re: Web Scraping Is Vital to Democracy
#48I use web scraping on my employer's Oracle EBS instance because coordinating with the battle-axe personalities that run ERP's is such a pain in the ass I would rather have a root-canal. With Web Scraping, I don't even have to talk to them.
For others: it looks like EBS in this context means "oracle e-business suite"
and ERP means "enterprise resource planning"
(maybe everyone else knows these, but just in case)
https://en.wikipedia.org/wiki/Oracle_Applications#Oracle_E-B...
Re: Web Scraping Is Vital to Democracy
#49I was trying to write a scraping script for a non profit to automate the work they were giving to law students to do manually earlier this year but the court system had agreed captchas that made it impossible. In my (legally uniformed) mind it felt like they were actively undermining people of lesser means by not allowing a simple script to run and essentially mandating an expensive legal team to do menial work
Re: Web Scraping Is Vital to Democracy
#50Earlier quoted context omitted.
Advertisements are harmful to every layer of society, mostly because they prey on you to instill desires that you probably wouldn't have had on your own (it being the case that this is their entire added value proposition). They should not be tolerated, and the fact that they ever were is a travesty. That being the case, any argument founded on "but advertisements" does not hold water.
It's not founded on "but advertisements" it's founded on "but paying for content". The fact that websites are developed by people (wages), hosted on infrastructure (hardware renting) and require countless other jobs is somehow magically forgotten in these discussions.
It's not up to me or anyone consuming content to help with figuring out the right way to pay for it. What is up to me is the choice to not be potentially exposed to malware any time I go to read the news.
While you may have a point, that point just doesn't matter within the context that we're talking about. Making companies money is not my responsibility.