Can anyone tell me which cloud provider they are using? I want to make sure that scrapinghub are on the list. I block the IP addresses of all the major cloud providers to prevent parasites such as this.
Web Scraping to Create Open Data
51–58 of 58 posts
Re: Web Scraping to Create Open Data
#52I had to use a similar approach when creating a cluster analysis of the amendments in the Italian Senate [0].
The Italian Senate offers a SPARQL endpoint [1], which unfortunately doesn't offer access to the texts of the amendments. So I had to roll my own and create a small spider for them using Scrapy [2].
[0]: https://github.com/jacquerie/senato.py/blob/master/analysis....
[2]: https://github.com/jacquerie/senato.py/blob/master/senato/sp...
Re: Web Scraping to Create Open Data
#53Re: Web Scraping to Create Open Data
#54Earlier quoted context omitted.
Yet companies like LexisNexis get most their data they resell this way.
Are they scraping copyrighted data? Or public records? Big difference.
No one in the US can hold copyrights to the pure 'facts', especially if one demonstrates they invested enough energy to 'creatively reinterpret' it. Scraping hasn't quite seen a Supreme Court ruling yet (@grellas correct me, please), but I'm sure one could make a reasonable argument that the energy invested in re-collating the data is sufficient enough to pass any barrier. See Feist Publications, Inc., v. Rural Telephone Service Co, 1991. and O'Connors opinion.
Re: Web Scraping to Create Open Data
#55Earlier quoted context omitted.
It's funny, everytime Weboob is presented somewhere, and everytime there is a post about the latest version of Weboob, the first comment is a variation of "it's sexist/boobs are unprofessional/grow up", and very very little time is spent talking about the actual thing, what it does and why its only goal is to become irrelevant. Sad thing. Here's what they have to say about it, and why there's very little chance they…
> It's funny, everytime Weboob is presented somewhere, and everytime there is a post about the latest version of Weboob, the first comment is a variation of "it's sexist/boobs are unprofessional/grow up", and very very little time is spent talking about the actual thing Seems like a bad name then, no?
I don't believe it's a problem with the name itself but more a problem with how commenters are more easily concerned about everything sounding professional even though it may very well have nothing to do with a job.
Re: Web Scraping to Create Open Data
#56Earlier quoted context omitted.
IANAL but in the EU at least, even databases comprised of simple "facts" are protected. It's a sad state of affairs when i'm not even allowed to scrape data generated using taxpayers' money, like the (required by EU laws) noise maps for cities, which I'd like to use to augment real estate offers, for example.
"Europe" would like to partially fund that noise database with income from businesses that use it. The result is less taxpayer money us needed. I think it's only the UK that has copyrightable fact databases
Except that doesn't happen because the last thing a new business idea needs is more red tape, paperwork and expenditure.
Re: Web Scraping to Create Open Data
#57Earlier quoted context omitted.
https://en.wikipedia.org/wiki/Copyright_infringement http://www.copyright.gov/title17/92chap5.html#501 https://www.law.cornell.edu/uscode/text/17/chapter-5 https://www.lib.purdue.edu/uco/CopyrightBasics/penalties.htm...
That's begging the question of whether Google's data on public streets is actually protected by copyright under U.S. law.
Re: Web Scraping to Create Open Data
#58Earlier quoted context omitted.
Are they scraping copyrighted data? Or public records? Big difference.
Facts aren't copyrightable. They scrape everything in the world they can get their hands on.
Collections of facts are: https://www.unc.edu/courses/2006spring/law/357c/001/projects...