Considering how much treasure has been poured into building recommendation engines for just about everything online, books have always been very difficult for me to find recommendations that work. Interested to try it!
Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
91–100 of 275 posts
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#92Neat! It's a validation of the model that 75%+ of the recommendations are things I've read and also enjoyed, with a few "read, didn't like" and some more "didn't read, don't really want to."
But I think to break the content-bubble effects to find the longer tail, some way to reject or blacklist things - and have that be taken into effect in the model - might help.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#93Echoing what everyone else has said here - awesome site, love how fast it was. I did notice that when I put in a single book in a series (in my case Going Postal, Discworld #33) that tended to dominate the rest of the selection. That does make sense, but I don't want recommendations for a series I'm already well into. Also noticed that a few books (Spycraft by Nadine Akkerman and Pete Langman, Tribalism is Dumb by An…
Yes I would say the handling of series is probably the biggest problem. Once my test metrics got to a point I was happy with and my quality spot checks passed (can I follow the models recommendations from one generic history book to Steven Runciman, also making sure popular books don't always dominate the results), I was ready to release because I had been working on this project for so long. The solution is probably…
Not just series, but I seem to mostly get a list of other books from the same authors.
The recommendations from other authors are good, but as far as I can tell I’ve read every single one of them.
Continuing to aggressively add everything it recommends eventually does seem to result in some interesting books I wasn’t familiar with, but I also end up with more and more books that are of zero interest to me.
For what it’s worth, I started with:
Infinite Jest David Foster Wallace
Europe Central William T. Vollmann
Gravity’s Rainbow Thomas Pynchon
White Noise Don DeLillo
One Hundred Years of Solitude Gabriel García Márquez
Very cool project though. I did end up ordering a couple of new books, so thank you very much.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#94See, now this is an excellent use of LLMs (if we're going to be using them at all). Low stakes if it gets shit wrong, but can provide some really useful and surprising answers! One request, it would be nice to not have to add Goodreads, since I don't use it. I've love to be able to enter a couple of book titles or an author and just get recommendations!
You don't have to import your Goodreads profile. You can type titles and authors in the box and find books to add to the list that way.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#95FYI, on this android tablet (android v12 / FF 144.0.2), the 'start typing a book title...' field doesn't do anything. On the Mac, it brings up a list of matches to select from.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#96I think this is cool and super fast -- kudos on whatever tech you needed to tackle to make it so.
I don't see anyone saying safety or ethics, so I'll just put it out there that it has some safety and ethical considerations you should consider.
Consider "inflammatory" books and how they could be used to harm a group of people. Although I recognize folks post this "publicly", I think the intersection feature provides more than Goodreads.
Let's say, people who have read "Mein Kampf" & "The Anarchists Cookbook" or some other combination that say "Antifa" to the current regime.
I'd recommend you have a list that you consider private, always and allow Users to add to that list so it's more scalable. If folks try to intersect with anything in that list, you can warn that you don't allow intersection with private books.
Anyway, super fun demo!
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#97Earlier quoted context omitted.
I'm surprised he got that much data. Goodreads uses several tricks to try to stop scrapers, for example pagination only works up to a few pages.
They might send him a bill for use of resources.
I’m wondering about how ethical it is to load down a resource in this way, open to opinions. There is a mention “I didn’t hammer down the servers” but what does that really even mean? The site isn’t being used as intended and just curious how other people feel about that.
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#98Amazing work. Do you plan on publishing the training code?
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#99I think this is cool and super fast -- kudos on whatever tech you needed to tackle to make it so. I don't see anyone saying safety or ethics, so I'll just put it out there that it has some safety and ethical considerations you should consider. Consider "inflammatory" books and how they could be used to harm a group of people. Although I recognize folks post this "publicly", I think the intersection feature provides m…
It always baffled me how we censor "Mein Kampf" but we - as a society - are super fine with either Alex Jones shouting about lasers and lizard people or Joe Rogan leaving an open mic to people claiming there are nuclear plants and space stations buried under pyramids [1].
Mein Kampf is absolutely terrible piece of literature not by it's message but by it's quality. It's exactly something I would expect to find in Alex Jones cell if we would sentence him to a year of solitary confinement.
[1] just a tiny exaggeration
Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model
#100I think this is cool and super fast -- kudos on whatever tech you needed to tackle to make it so. I don't see anyone saying safety or ethics, so I'll just put it out there that it has some safety and ethical considerations you should consider. Consider "inflammatory" books and how they could be used to harm a group of people. Although I recognize folks post this "publicly", I think the intersection feature provides m…
I don't want "safety" or "ethics" if the requirements for them are banning or hiding books based on somebody's ideology whether or not it agrees with mine.