151–160 of 196 posts
Earlier quoted context omitted.
Man your comment is hilarious because in fact Amazon DOES provide an API for exactly that
And yet... > We've once had to scrape Amazon product pages because they have a lot of API endpoints, but those didn't contain the data we needed. ...only a couple of comments up.
Earlier quoted context omitted.
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
It is copyright information, no? So technically it is intellectual property theft if the scraping use is for commercial purposes.
If you build a database of touristic places and display in your website, the information is not protected by copyright.
In Europe they have laws covering _sui generis database rights_, but they are from another era and unenforceable nowadays.
Earlier quoted context omitted.
I think there's an opportunity for a new JS framework to have something like randomly generated dom that will always display the page and elements the same to a human but constantly break paths for computers. Like displaying a table with semantic elements, then divs, then using an iframe with css grid and floating values over the top. This almost seems like a problem for AI to solve.
To some extent those already exist and I get annoyed by them when they cause 1Password to be useless on their login page. But it probably would help with algorithmic scraping.
Earlier quoted context omitted.
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
> You're going to need to explain how scraping publicly available information on a website is theft. Seriously? Do I need to explain why a song doesn’t enter the public domain when it is played on the radio?
Earlier quoted context omitted.
> You're going to need to explain how scraping publicly available information on a website is theft. Seriously? Do I need to explain why a song doesn’t enter the public domain when it is played on the radio?
No but those are substantively different situations such that this exact thing is being argued in the highest courts of the US. It's not quite the cut and clear case you seem to believe it to be.
Earlier quoted context omitted.
No but those are substantively different situations such that this exact thing is being argued in the highest courts of the US. It's not quite the cut and clear case you seem to believe it to be.
I'm not American. I'm interested to read about these cases. Can you point me towards some relevant material, or at least cite the case name(s)?
Earlier quoted context omitted.
I'm not American. I'm interested to read about these cases. Can you point me towards some relevant material, or at least cite the case name(s)?
Ah yup, certainly. So this is the one I was mainly thinking of -- https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
You appear to have claimed that rights over material is broadly relinquished if it's published in public:
> If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website
And your distinction was further clarified when you argued that placing barriers to access fundamentally changes the equation:
> Note, a simple sign up being required to view a website makes it not publicly available information any longer and you can cover usage, again, in a license.
Perhaps you meant to speak only of material which is not subject to copyright? In which case I think your argument does track.
Earlier quoted context omitted.
Ah yup, certainly. So this is the one I was mainly thinking of -- https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
Interesting. From what I can gather, the material in HiQ Labs v. LinkedIn was not claimed to be under copyright and that the argument being fought over in court was with respect to mechanically subverting access to a competitor. You appear to have claimed that rights over material is broadly relinquished if it's published in public: > If information is your competitive advantage maybe you shouldn't have it on a publi…
I believe the future will make us more free by using more bot / AI technology since who wants to spend their whole day in front of a computer and research information if machines can do the job just fine?
Earlier quoted context omitted.
And how do you get your 'inventory data'? Aren't you scraping (or using scraped data) yourself? Oh the irony :)
No, we're a contracted provider for these customers. They ingest their data into our network through APIs or CSVs.