Whatever I put in, it wants me to read Sapiens :_(
Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
21–30 of 275 posts
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#22I've worked in recommender systems for a while, and it's great to see them publicized.
SASRec was released in 2018 just after transformer paper, and uses the same attention mechanism but different losses than LLMs. Any plans to upgrade to other item/user prediction models?
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#23I gave up on goodreads reviews. I've been burned too many times by highly rated books that weren't that good. If you're into (horny) ya romance fantasy then goodreads is great, but it's not for me. I haven't really found a substitute.
Any broadly used ratings system is total garbage. Goodreads ratings, Google Maps ratings, Amazon reviews, Vivino for wine, et cetera. Even assuming the reviews are real and genuine, most people just aren’t good at writing reviews, and the handful that are often have wildly different criteria than you. Someone already commented with one enthusiast site - and sure, enthusiast sites are often better than the mainstream option (see also: CellarTracker for wine) but honestly my advice is to get good at determining the quality of the thing yourself. For books there are a ton of hints about what you’ll be getting. “NYT Bestseller”, “xyz book club”, certain publishers, who’s quoted on the back, when was it published, who wrote it? All of those things can help you rapidly identify books. I personally dislike most modern books and prefer the “classics”, so a lot of this is only useful as a negative signal, but even then there are positive signals, for example a reference to a much older book.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#24I love this site, and the approach! Great seeing someone making good use of Goodreads data.
Sadly my experience with the book recommender isn't too great because of the 64 book limit. If I import either the most recent or least recent 64 book, 95% of the books it recommends to me are books I've read. Though it was helpful for spotting a few books I've read that I didn't log on Goodreads. Guess I'm pretty consistent.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#25Interesting. I tested it with sci-fi, and it definitely recommends good books, but not sure how accurate it is at surfacing the sub genres / themes. For example for [aurora -ksr, seveneves, project hail mary, ender's game] it gave me dune. Which is a great book, but not in the "first-ish contact" style I hoped it would be. Another thing I noticed is that it tends to recommend 2nd and 3rd books in a series, which is a…
Came here to say this (recommending book 2 and 3 in a trilogy). Great app otherwise!
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#26I'm impressed it recommended so many books i've already read and liked! I have a big reading backlog but once it's whittled down I will likely come back to this. One feature request would be to also show a "why this is recommended" for each recommendation so I can further narrow down the list for what I'm looking for
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#27The recommendations are pretty good; even though I only input six books, it was enough for it to recommend books I have on my wish list. Definitely going to play around some more. Plus, the website is super fast, very impressive. Any chance we could get an API going at some point? Are you planning to open source the work? I'm interested in the scrapping of Goodreads too. I'm building a book metadata aggregation API a…
Thank you for the compliments :) I used 50-100 datacenter proxies. I just logged requests made by the iOS app with Charles and then recreated the headers to the best of my ability though the server did not seem to be very strict at all. Worth noting though that static residential proxies are not too expensive these days anyways. Re the API: The model does actually run fairly well on CPU so it probably wouldn't be too…
Speaking of sustained scraping for AI services, I found a strange file on your site: https://book.sv/robots.txt. Would you be able to explain the intent behind it?
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#28I've worked in recommender systems for a while, and it's great to see them publicized. SASRec was released in 2018 just after transformer paper, and uses the same attention mechanism but different losses than LLMs. Any plans to upgrade to other item/user prediction models?
I'm not an expert by any means but as far as sequential recommendations go, aren't SASRec and its derivatives pretty much the name of the game? I probably should have looked into HSTUs more. Also this / sparse transformers in general: https://arxiv.org/pdf/2212.04120
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#29I don't know. I entered, trying to be popular but at least slightly? opiniated:
Tigana, Hyperion, A Fire Upon the Deep, Blindsight, Moby Dick
and I got a list. Sure, read all that or wasn't interested for reasons, I added (only Neuromancer on initial recommendations):
Neuromancer, VALIS, Quantum Thief, Towing Jehovah.
List did not get more interesting.
Book recommendations are still kind of difficult.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#30Like the idea! Wondering: Weren’t the early LLMs trained on data in Goodreads as well? I can upload and ask ChatGPT as well, and it will give me similar recommendations, no?