I entered "Alone Together: Why We Expect More from Technology and Less from Each Other" and I received books about Steve Jobs, Harry Potter and "The Subtle Art of Not Giving a F*ck". Like how???
Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
41–50 of 275 posts
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#42Can you share the details about the Meilisearch instance? How big is the box and database size?
Everything (namely Meilisearch, Postgres and the web server in Go) besides the model inference is running on a Hetzner server with a large SSD and an "AMD Ryzen 7 3700X 8-Core Processor." The data.ms directory is about 40GB. Once the HN traffic dies down I will probably move the model back to the Hetzner server so I don't have to pay $0.15/hour for an A4000.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#43I gave up on goodreads reviews. I've been burned too many times by highly rated books that weren't that good. If you're into (horny) ya romance fantasy then goodreads is great, but it's not for me. I haven't really found a substitute.
GR is also great if you are into academic nonfiction, Classics, poetry, etc. The site does, after all, let you track and review any publication with an ISBN. What my peers and I use it for is worlds apart from the romance novel or LGBT young-adult book reviewing community that often puts GR in the news, and far away from all the drama that rages around genre fiction.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#44I've worked in recommender systems for a while, and it's great to see them publicized. SASRec was released in 2018 just after transformer paper, and uses the same attention mechanism but different losses than LLMs. Any plans to upgrade to other item/user prediction models?
I'm not an expert by any means but as far as sequential recommendations go, aren't SASRec and its derivatives pretty much the name of the game? I probably should have looked into HSTUs more. Also this / sparse transformers in general: https://arxiv.org/pdf/2212.04120
There's a few alternatives, but SASRec is a good baseline for next-item recommendation. I'd look at BERT4Rec too. HSTU is definitely a strong step forward, but stays in the domain of ID models. HSTU also seems to rely heavily on some extra item information that SASRec does not (timestamps).
Other models include Google's TIGER model which uses a VAE to encode more information about items. Similar to how modern text-to-voice operates.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#45I entered "Alone Together: Why We Expect More from Technology and Less from Each Other" and I received books about Steve Jobs, Harry Potter and "The Subtle Art of Not Giving a F*ck". Like how???
If you want recommendations solely based on one book, please try the similar page: https://book.sv/similar?id=13566692
These seem to fit the description you are going for better. The model is trained to predict the next book in the sequence. Those other books you listed happen to be very popular, so in the absence of information about you (only having 1 book), the model will tend to recommend those.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#46Where do nice scrapes like this end up? Are there BitTorrents out there for scrapes like this?
Honestly this would finally be the web2.0 we all wanted & hoped for. It's against majesty that it's all captured owned user content that is legally captured by essentially public message boards/sites.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#47I unexpectedly liked this. I thought the recommendations were actually useful.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#48Care to share the scrapped data? I would love to play around with it.
I'm surprised he got that much data. Goodreads uses several tricks to try to stop scrapers, for example pagination only works up to a few pages.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#49I entered "Alone Together: Why We Expect More from Technology and Less from Each Other" and I received books about Steve Jobs, Harry Potter and "The Subtle Art of Not Giving a F*ck". Like how???
> Provide 3+ books for best results.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#50Echoing what everyone else has said here - awesome site, love how fast it was.
I did notice that when I put in a single book in a series (in my case Going Postal, Discworld #33) that tended to dominate the rest of the selection. That does make sense, but I don't want recommendations for a series I'm already well into.
Also noticed that a few books (Spycraft by Nadine Akkerman and Pete Langman, Tribalism is Dumb by Andrew Heaton) that I know are in goodreads and reviewed didn't show up in the search. I tried both author's name and the title of the book. Maybe they aren't in the dataset.
It did stumble with some books more niche books (The Complete Yes Minister). Trying the "Similar" button gave me more books that were _technically_ similar because they were novelizations of British comedy shows, but not what I was looking for.
For more common books though it lined up very well with books already on my wishlist!