Show HN: I scraped 3B Goodreads reviews to train a better recommendation model • Hacker News Board

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#11

Post by noir_lord » Thu, Nov 06, 2025, 8:42 PM UTC

It has a tendency to recommend books in the same series as are input (putting aside that if I like a book in a series I've likely already read the series).

It did suggest Murderbot Diaries (not on the input but a series I have read and did like) and an Adrian Tchaikovsky I hadn't read :).

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#12

Post by jamesponddotco » Thu, Nov 06, 2025, 8:45 PM UTC

comrade1234 wrote: post #10

I gave up on goodreads reviews. I've been burned too many times by highly rated books that weren't that good. If you're into (horny) ya romance fantasy then goodreads is great, but it's not for me. I haven't really found a substitute.

I'm not into the social aspect, so Goodreads was never an option, but Hardcover[1] seems like a pretty good alternative.

[1]: https://hardcover.app

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#13

Post by costco » Thu, Nov 06, 2025, 8:47 PM UTC

jamesponddotco wrote: post #6

The recommendations are pretty good; even though I only input six books, it was enough for it to recommend books I have on my wish list. Definitely going to play around some more. Plus, the website is super fast, very impressive. Any chance we could get an API going at some point? Are you planning to open source the work? I'm interested in the scrapping of Goodreads too. I'm building a book metadata aggregation API a…

Thank you for the compliments :) I used 50-100 datacenter proxies. I just logged requests made by the iOS app with Charles and then recreated the headers to the best of my ability though the server did not seem to be very strict at all. Worth noting though that static residential proxies are not too expensive these days anyways.

Re the API: The model does actually run fairly well on CPU so it probably wouldn't be too expensive to serve. I guess if there is demand for it I could do it. I think most social book sites would probably like to own their recommendation system though.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#14

Post by bananaflag » Thu, Nov 06, 2025, 8:48 PM UTC

noir_lord wrote: post #11

It has a tendency to recommend books in the same series as are input (putting aside that if I like a book in a series I've likely already read the series). It did suggest Murderbot Diaries (not on the input but a series I have read and did like) and an Adrian Tchaikovsky I hadn't read :).

Yeah the hardest problem for recommendation systems is to find non-Star Wars books which are like some specific Star Wars books and unlike some other Star Wars books. I would say it's AGI-complete ;)

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#15

Post by walthamstow » Thu, Nov 06, 2025, 8:51 PM UTC

Works pretty well with cookbooks. Very cool work.

One suggestion would be to make the search less strict on diacritics. Searching for popular cook J. Kenji López Alt was only successful if I entered the correct O.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#16

Post by costco » Thu, Nov 06, 2025, 8:51 PM UTC

noir_lord wrote: post #11

It has a tendency to recommend books in the same series as are input (putting aside that if I like a book in a series I've likely already read the series). It did suggest Murderbot Diaries (not on the input but a series I have read and did like) and an Adrian Tchaikovsky I hadn't read :).

It's explicitly trained to predict the next book read in a sequence, which is why you get that behavior. There's probably a better way for me to handle it rather than having 5 books from the same series tend towards the top though.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#17

Post by qingcharles » Thu, Nov 06, 2025, 8:59 PM UTC

aj_hackman wrote: post #5

Thank you! Because of this, "The Making of Prince of Persia: Journals 1985–1993" by Jordan Mechner is on its way to my house.

You definitely will not regret that purchase. It's a very enjoyable read.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#18

Post by NitpickLawyer » Thu, Nov 06, 2025, 9:01 PM UTC

Interesting. I tested it with sci-fi, and it definitely recommends good books, but not sure how accurate it is at surfacing the sub genres / themes. For example for [aurora -ksr, seveneves, project hail mary, ender's game] it gave me dune. Which is a great book, but not in the "first-ish contact" style I hoped it would be.

Another thing I noticed is that it tends to recommend 2nd and 3rd books in a series, which is a bit so-so. If I add the first book in a series, I probably already read the whole series...

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#19

Post by qingcharles » Thu, Nov 06, 2025, 9:02 PM UTC

I put in a bunch of books and hit recommendations and... I'd already read 95% of them, so at least we know it works well! (checking out the other 5% now)

p.s. one idea: when you click [Add] on the recommended books list, it should remove it from that list

p.p.s. if there is a way to filter out the spam "Summary of ____" books, that would be good too

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#20

Post by yoz-y » Thu, Nov 06, 2025, 9:03 PM UTC

It works pretty well in the sense that after inputting only a few quite diverse books it gave me recommendations for a lot of books that I’ve already also read and enjoyed.

I would also really like a possibility to add negative signal. It did also recommend books that seemed interesting to me but I ultimately didn’t like.

Overall quite impressive.