Live data from Hacker News

Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

book.sv

181–190 of 275 posts

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#181

Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…

I’m not taking sides in this debate, however since feeding whole books into LLMs is considered legal fair use now, I guess these reviews don’t require a permission as well. Would be great to hear a professional lawyer take on this.

The hidden gotcha in the Anthropic judgement (which I think is what you’re referencing?) is that feeding whole books into LLMs is considered legal fair use if you obtain them legitimately.

I suspect we need to wait for the NYT (and others) case to be decided before we know whether scraping sites in contravention of their terms is also fair use for LLM training.

My own opinion (as someone who creates written content on an occasional professional basis) is that if you can’t monetise your content in some other way than blocking people from accessing it then your content probably isn’t as valuable as you think.

But at the same time that’s tricky when it’s genuine journalism, as in NYT’s case.

Obviously user generated content reviewing books online is rather different because the motivation of the reviewers was (presumably) not to generate money. And, indeed, with goodreads there’s a strong argument that people have already been screwed over after their good faith review submissions were packaged up as an asset and flogged to Amazon. A lot of people were quite upset by that when it happened a decade or so back.

So from a ‘moral arguments’ perspective I don’t think scraping goodreads is as problematic as other scraping examples.

(Sorry, none of this was aimed at you - your comment just got me thinking and it seemed as good a place as any to put it!)

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#184
post #179

Earlier quoted context omitted.

It is the future: I own nothing, and I've never been happier. They can sue me and take nothing.

I've been trying to convince myself I'd be able to live like Diogenes, sleep in the streets, bathe in the sea and just generally survive off scraps - but I think that only works if other people can afford to throw away scraps.

If you live in the west it’s no problem - the amount of waste there is insane

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#186

Earlier quoted context omitted.

Not only am I taking 1,000 apples, but I use those 1,000 apples to start my own orchard and encourage people to come to it instead of yours.

... and somehow your garden did not lose any apple in the process.

But you are an apple seller

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#187

Earlier quoted context omitted.

Not only am I taking 1,000 apples, but I use those 1,000 apples to start my own orchard and encourage people to come to it instead of yours.

Yeah but if I program a drone swarm to automate this process it’s for the greater good — more apples for everyone! And I only charge a tiny subscription for access to all my drone-managed orchards, you can eat as many apples as you want. But don’t steal any and start your own orchard or I sue.

All the people who care for the trees and pick the apples have lost their job while an apple became nearly worthless, but without a job it‘s still unaffordable.

Replace your drones with China or India and you have the current situation in the US.

Apple farmers go out of business so you lose the people who create new varieties.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#188

Earlier quoted context omitted.

It is truly criminal that such a bright and brilliant model of ethics, Amazon, should endure such an attack.

Unethical behavior does not become good just because it happens to hurt "bad people" (or more accurately, companies bought by bad people).

Using a sword to stab someone is evil, therefore, stabbing someone who is stabbing me with a sword is evil?

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#190

Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…

What expectation of confidentiality are you ascribing to people having posted publicly accessible opinions on the internet? Out of curiosity, is your point about TOS out of concern for the poster or for Goodreads?

What does that comment have to do with confidentiality?
Post reply on HN