Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

book.sv

Post reply on HN
Print view View original

171–180 of 275 posts

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#171

Post by jalk »

croes wrote: post #158
pantropy wrote: post #143

Earlier quoted context omitted.

Technically speaking none of Goodreads material or content is being used publically, the only information displayed on the site is freely available (Title, Author) and not Goodread's property. You could try to argue that this falls under "create derivative works from any materials or content accessible on the Service" but even then it seems really flimsy to say that recommending books based on Goodread reviews is an…

I visit your garden and take 1 apple from your tree I visit your garden and take 1000 apples from your tree. Not that different.

Not a great analogy, since a digital copy leaves the original intact unlike your apples

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#174

Post by voidUpdate »

MichaelBosworth wrote: post #165
voidUpdate wrote: post #124

Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…

What expectation of confidentiality are you ascribing to people having posted publicly accessible opinions on the internet? Out of curiosity, is your point about TOS out of concern for the poster or for Goodreads?

My expectation isn't of confidentiality, but of attribution. Sure, my website is perfectly accessible on the internet, and I'm fine with being able to find it on google, but if you pipe it into an algorithm that will start throwing out stuff based on what I wrote, with zero reference to me at all, I'd get a bit annoyed. This website has taken the combined output of probably thousands of people, shoved it into an algorithm and is then using their work to give "original" ideas. If one person wanted their content removed from the system, how would you do that?

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#176

Post by sputr »

diffeomorphism wrote: post #141

The robots.txt is pretty explicit that this scraping is "disallowed" https://www.goodreads.com/robots.txt So legalities aside, this seems unethical.

Why would it be unethical?

This obsession with "everything must be commercialized" is really killing creativity.

Now if the author was commercializing other peoples reviews, sure, it's potentially(!) unethical. But scraping a website for reviews that are publicly(!) posted, training a recommendation LLM and then sharing it, for free, seems ... exactly the ideal use case for this technology.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#177

Post by diffeomorphism »

paulnpace wrote: post #168
diffeomorphism wrote: post #141

The robots.txt is pretty explicit that this scraping is "disallowed" https://www.goodreads.com/robots.txt So legalities aside, this seems unethical.

It is truly criminal that such a bright and brilliant model of ethics, Amazon, should endure such an attack.

Unethical behavior does not become good just because it happens to hurt "bad people" (or more accurately, companies bought by bad people).

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#178

Post by mejutoco »

kemotep wrote: post #161
croes wrote: post #158

Earlier quoted context omitted.

I visit your garden and take 1 apple from your tree I visit your garden and take 1000 apples from your tree. Not that different.

Not only am I taking 1,000 apples, but I use those 1,000 apples to start my own orchard and encourage people to come to it instead of yours.

... and somehow your garden did not lose any apple in the process.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#179

Post by exe34 »

paulnpace wrote: post #167
exe34 wrote: post #160

Earlier quoted context omitted.

it's only legal if you have a team of lawyers though. the law still applies to the rest of us.

It is the future: I own nothing, and I've never been happier. They can sue me and take nothing.

I've been trying to convince myself I'd be able to live like Diogenes, sleep in the streets, bathe in the sea and just generally survive off scraps - but I think that only works if other people can afford to throw away scraps.
Post reply on HN