The robots.txt is pretty explicit that this scraping is "disallowed"
https://www.goodreads.com/robots.txt
So legalities aside, this seems unethical.
Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
141–150 of 275 posts
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#142I feel that the last added book in one's list seem to have more influence on the recommendations, which results in a rather similar type of recommendations.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#143Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…
Technically speaking none of Goodreads material or content is being used publically, the only information displayed on the site is freely available (Title, Author) and not Goodread's property.
You could try to argue that this falls under "create derivative works from any materials or content accessible on the Service" but even then it seems really flimsy to say that recommending books based on Goodread reviews is an infringemnt.
It's just not that different to a youtuber saying "I read reviews for 50 books, here's the ones to read"
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#144Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…
Why ask questions you already know the answers to?
Because some tech adjacent people still have morals?
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#145Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…
Technically speaking none of Goodreads material or content is being used publically, the only information displayed on the site is freely available (Title, Author) and not Goodread's property. You could try to argue that this falls under "create derivative works from any materials or content accessible on the Service" but even then it seems really flimsy to say that recommending books based on Goodread reviews is an…
I'd be impressed if a youtuber could read 3 billion reviews and recommend books to you based on that
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#146The robots.txt is pretty explicit that this scraping is "disallowed" https://www.goodreads.com/robots.txt So legalities aside, this seems unethical.
I agree. As a frequent reviewer on Goodreads, this feels really icky.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#147Considering how much treasure has been poured into building recommendation engines for just about everything online, books have always been very difficult for me to find recommendations that work. Interested to try it!
I've made multiple attempts to use "AI" to find the names of books I've read in the past. Never managed to find a single one. Quite disappointed with that. I've read so many books that I have no idea what were called.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#148Thank you! Because of this, "The Making of Prince of Persia: Journals 1985–1993" by Jordan Mechner is on its way to my house.
Here’s a human recommendation if you like that you may like these that I’ve read:
1.Sid Meier’s Memoir!: A Life in Computer Games — Sid Meier 2.Source Code: My Beginnings — Bill Gates 3.Build: An Unorthodox Guide to Making Things Worth Making — Tony Fadell 4.Prince of Persia: The Journals — Jordan Mechner 5.A Theory of Fun for Game Design — Raph Koster 6.Ask Iwata: Words of Wisdom from Satoru Iwata, Nintendo’s Legendary CEO — Viz Media (Editor) 7.Control Freak: My Epic Adventure Making Video Games — Cliff Bleszinski 8.Once Upon Atari: How I Made History by Killing an Industry — Howard Scott Warshaw 9.Press Reset: Ruin and Recovery in the Video Game Industry — Jason Schreier 10.Masters of Doom: How Two Guys Created an Empire and Transformed Pop Culture — David Kushner
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#149I feel that the last added book in one's list seem to have more influence on the recommendations, which results in a rather similar type of recommendations.
This is a result of the use of positional embeddings, which typically results in the final item being weighted very highly. The problem is that this information is shown to be very relevant to the task of predicting the next item interacted with. If you add more books the effect of this is somewhat diluted.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#150I typed "Introduction to Real Analysis by Bartle" and I got: Steve Jobs by Walter Isaacson Harry Potter and the Sorcerer's Stone (Harry Potter, #1) by JK Rowling Topology by James R Munkres and so on.. Munkres' book is relevant and I want to read it, but what have Steve Jobs and Harry Potter got to do with with mathematics?
You may find these better:
https://book.sv/similar?id=211570
> Note 1: If you only provide one or two books, the model doesn't have a lot to work with and may include a handful of somewhat unrelated popular books in the results. If you want recommendations based on just one book, click the "Similar" button next to the book after adding it to the input book list on the recommendations page.