I have a web app https://bookhive.buzz which is a GoodReads alternative based on BlueSky’s protocol. I scrape all of the book data from Goodreads too.
I would love to be able to add a recommendation system based on this.
Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
71–80 of 275 posts
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#72Two bugs to know about. First, you are using a deprecated API call that fails in Firefox. Second, you are using an HTTP endpoint that fails to upgrade to HTTPS to call the GoodReads API, which also fails with HTTPS-Only enabled in both Chrome and Firefox.
The idea seems good, but since I can't import my GoodReads successfully, it's hard for me to try
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#73Awesome site and speed!
My advice from someone who has built recommendation systems: Now comes the hard part! It seems like a lot of the feedback here is that it's operating pretty heavily like a content based system system, which is fine. But this is where you can probably start evaluating on other metrics like serendipity, novelty, etc. One of the best things I did for recommender systems in production is having different ones for different purposes, then aggregating them together into a final. Have a heavy content-based one to keep people in the rabbit hole. Have a heavy graph based to try and traverse and find new stuff. Have one that is heavily tuned on a specific metric for a specific purpose. Hell, throw in a pure TF-IDF/BM25/Splade based one.
The real trick of rec systems is that people want to be recommnded things differently. Having multiple systems that you can weigh differently per user is one way to be able to achieve that, usually one algorithm can't quite do that effectively.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#74Very neat. Even found a couple Cold War-setting books to read and an entire series of 6 books on the same topic, All from searching up Team Yankee.
Thanks for the new reading list :D
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#75It works pretty well in the sense that after inputting only a few quite diverse books it gave me recommendations for a lot of books that I’ve already also read and enjoyed. I would also really like a possibility to add negative signal. It did also recommend books that seemed interesting to me but I ultimately didn’t like. Overall quite impressive.
[deleted]
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#76It seems to work decently even with just one or two titles for popular titles, but less so for the niche.
For example, the title "Impro: Improvisation and the Theatre" by Keith Johnstone, linked by another article posted to HN today gives back the following suggestions:
- Truth in Comedy: The Manual of Improvisation by Charna Halpern - Steve Jobs by Walter Isaacson - 1984 by George Orwell - Harry Potter and the Sorcerer's Stone (Harry Potter, #1) by J.K. Rowling - Sapiens: A Brief History of Humankind by Yuval Noah Harari - The Alchemist by Paulo Coelho - The Tipping Point: How Little Things Can Make a Big Difference by Malcolm Gladwell - Dune (Dune, #1) by Frank Herbert
It's a bit unfortunate that all suggestions are fairly popular titles, which are fairly easy to find, while the unpopular or niche may be just as well written but a lot harder to find.
Within niche topics or books, it is also usually harder to provide multiple similar enough titles up front.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#77The How it works it way too short :) I'd love to see some scripts, know the hardware you use, etc...
By the way you could use Summa FTS Wasm + Duckdb Wasm to have the same website without any backend except file hosting. Maybe even just Duckdb Wasm with it's FTS would be enough. Summa FTS is very similar to meilisearch in essence because they're both derived from Tantivy.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#78It seems to work decently even with just one or two titles for popular titles, but less so for the niche. For example, the title "Impro: Improvisation and the Theatre" by Keith Johnstone, linked by another article posted to HN today gives back the following suggestions: - Truth in Comedy: The Manual of Improvisation by Charna Halpern - Steve Jobs by Walter Isaacson - 1984 by George Orwell - Harry Potter and the Sorce…
It's recommended that you put at least 3 books in. If you would like recommendations just based on one book, click the similar button on the book, it should take you to this page: https://book.sv/similar?id=297914
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#79I would love to be able to filter the resulting list by removing certainly all books that in the same series but I think removing all books by authors that I have already listed would be great to get new things that I haven't already read. The resulting recommendations maybe included 1 new book for me.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#80The How it works it way too short :) I'd love to see some scripts, know the hardware you use, etc... By the way you could use Summa FTS Wasm + Duckdb Wasm to have the same website without any backend except file hosting. Maybe even just Duckdb Wasm with it's FTS would be enough. Summa FTS is very similar to meilisearch in essence because they're both derived from Tantivy. https://izihawa.github.io/summa/quick-start/
I use a Hetzner server with Ryzen 7 3700X and an SSD.
I think I could get the model to work with ONNX web but it'd be a 2GB download so the user experience wouldn't be too great. My Meilisearch index is ~40GB but I don't know how much that could be compressed down.
Here's how the similar page for books is generated, which I forgot to mention on the "how it works" page: https://gist.github.com/chris124567/8d06d64bfe827cb7f6121f93...