41–50 of 205 posts
I am a Goodreads employee. this is a burner account. i could be fired for posting. we love Goodreads and we know it is bad today. we are working to fix it. there are very few of us. we are trying. we are making big changes soon.
How do people build recommendation engines? Like, even when you have access to the site's entire API and can write your own client for it, there's still the fact that their recommendations are generally better. It sounds like an extremely important value-add. There are many sites that I will only use the app for because that is where the recommendations are shown. But to me taking a user profile and an article and sp…
One method, collaborative filtering with latent factor analysis, popularized by its efficacy in making recommendations on Netflix, is to use matrix multiplication to solve the problem.
E.G. Let’s say you have all users (rows) x all books (columns), in a massive sparsely populated matrix, where the value is the rating that user gives a book.
To make recommendations, the goal is to “guess“ what a user would rate a book they haven’t read, and if your guess is they would give it 5 stars, and then you recommend it, and the user gives it 5 stars, it’s a good recommendation.
The “latent factor” idea is breaking the problem up, so in order to compute the rating matrix that is the final size N users by M books, you split it into:
N users x D latent factor (cross product) D latent factor x M books = N users x M books
It then becomes a machine learning problem, using a loss function plus gradient descent, to solve for that D latent factor.
Customers with a similar latent factor, will have similar taste.
Once you have the latent factors, you find the nearest neighbors (the closest other latent factor vectors measured by dot product or cosine similarity), to compute the nearest books. The vectors that multiply together to give the highest rating, will be the best recommendations for the user.
Open Library is an free, online California Library with millions of digital books to read and borrow. It's additionally an open catalog of millions of books which you may track like goodreads.
Best part? If it's not to your liking, the whole project is open source! https://github.com/internetarchive/openlibrary
Open Library has a very strong volunteer tribe of book-loving librarians, developers, and designers working to make the project better for our community.
We meet once a week at 11:30am PT. Ask me for the link: mek+ol@archive.org
RIGOROUS BOOK RECOMMENDATIONS: We're also cultivating a 2nd non-profit open source experiment called TheBestBookOn.com (it leverages Open Library's catalog) which allows book-lovers to ask for or make rigorous book recommendations:
https://github.com/Open-Book-Genome-Project/TheBestBookOn.co...
It's a very early prototype (we discuss development during our weekly Open Library call). It's being organized by Lauren Milliken, Aasif Khan, and myself and we'd love more contributors to join the discussion: mek+bbo@archive.org
Howdy, Mek here -- I run https://openlibrary.org over at the non-profit Internet Archive (the folks that bring you the Wayback Machine). Open Library is an free, online California Library with millions of digital books to read and borrow. It's additionally an open catalog of millions of books which you may track like goodreads. Best part? If it's not to your liking, the whole project is open source! https://github.co…
https://www.lesswrong.com/posts/xg3hXCYQPJkwHyik2/the-best-t...
Recommendations are better when you have enough info to draw a line rather than consider a single point.
There are many websites online which give you detailed reviews. But how does their review compare to other similar books? And has this reader also read those books?
Thebestbookon.com requires a reviewer to have read 3 books and justify choosing one of them as a winner (like a college basketball bracket).
If they're using collaborative filtering, there probably isn't a simple explanation. Basically, you feed in a list of ", , " and the algorithm generates a feature vector for each book and each user. So the reason some book gets recommended is because... the dot product of that book's vector with your vector was high. You can do this easily with off-the-shelf libraries, like Surprise[1] (I'm using that lib in my startup[2]).
At least this is what happens in matrix factorization methods. Recommendations from k-nearest neighbor methods can be explained more easily, but knn doesn't scale as well.
This is a minor drawback of matrix factorization--people have been shown to trust recommendations more if they understand how the recommendations were generated. Twitter recently published a super interesting paper[3] about how they generate recommendations at scale, and as a bonus their method is explainable. They create a model that describes the communities in Twitter--groups of people who follow the same set of influencers. Users and items are represented by vectors, where each element of a vector describes to what degree a user/item fits in a certain community. When generating recommendations, you can get the user's top communities and then fetch items for those communities. If you generated a text description for each community, you could include that with each recommendation.
[1] https://github.com/NicolasHug/Surprise
[3] https://www.kdd.org/kdd2020/accepted-papers/view/simclusters...
Good lord what I wouldn’t give for a better goodreads. Never has so much useful and actionable data been squandered. For example, I could write pages about how bad the “top books” lists are. The Goodreads Choice Awards are purely and literally popularity contests, for example. Why would this be the case - you have millions of user ratings, you should be using those to surface exciting and unknown books rather than th…
Would you share the titles and authors of those two books?
I'm skeptical of this person's ability to "know" what they don't want to read. E.g., they are recommended Shoe Dog and a King novel, which they claim they don't want to read. However, if other people have read similar books and rated them similarly to this person and have read Shoe Dog and that King novel and rated those well, then this person may like those books. It seems they are assuming they won't like them, but…
Huh, I get recommended Murakami a lot but really didn't enjoy Colorless - maybe I'll try a different one.
Earlier quoted context omitted.
What are the most important changes that your team is prioritizing? What are your biggest challenges with making them?
I want to be specific but I would get fired if they saw. but we read EVERY post about us. we are being ambitious and big changes are coming. biggest challenge is there is too much to do and we have so few staff.
I'm skeptical of this person's ability to "know" what they don't want to read. E.g., they are recommended Shoe Dog and a King novel, which they claim they don't want to read. However, if other people have read similar books and rated them similarly to this person and have read Shoe Dog and that King novel and rated those well, then this person may like those books. It seems they are assuming they won't like them, but…
Huh, I get recommended Murakami a lot but really didn't enjoy Colorless - maybe I'll try a different one.
Blog post from yesterday: https://robbie.antenesse.net/2020/09/11/one-year-of-readlebe...
Edit: oh, there is also Bookwyrm, as mentioned in this megalist: https://git.feneas.org/feneas/fediverse/-/wikis/watchlist-fo...