Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

book.sv

Post reply on HN
Print view View original

231–240 of 275 posts

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#231

Post by idoubtit »

costco wrote: post #159
idoubtit wrote: post #121

Earlier quoted context omitted.

The "Intersect" page was useless for me. I added 15 books, but got no matching user. I entered a cycle of removing-searching, and at 10 books I had 2 users: one had read 41353 books, and the other 85363, with no ratings... To be useful, the "Intersect" page should have: - find near matches when there is no exact match with every book, - ignore fake users (can any human read 80k books in many languages?), - do not ign…

What do you think the probability that someone else read 15 books you also read is? It’s very unlikely unless they are all staples of a genre, part of the same series, or just extremely popular in general. 3-5 books is how much I would use on that page. I have found interesting accounts of medievalists, people who work at think tanks, etc with it. Fake users I would agree should be filtered, but I don’t think filteri…

I wanted to find users that loved the same kinds of classical novels. The core of my list was each famous work of famous classical writers like Dostoievsky, Tolstoi, Huxley and Borges. I added a few excellent authors, still famous but to a lesser degree, like Italo Calvino or Marguerite Yourcenar. I know there are many readers of the whole list I wrote, I could name a few among my friends and family.

So I think the problem was not in the existence of similar readers, but in the way to reach them. Few people that read classical books log in Goodreads (I don't) and even fewer input what they've read over the past decades.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#232

Post by colechristensen »

cyrusradfar wrote: post #112
colechristensen wrote: post #100

Earlier quoted context omitted.

I don't want "safety" or "ethics" if the requirements for them are banning or hiding books based on somebody's ideology whether or not it agrees with mine.

I'm not asking him to hide the book recommendations, I'm talking about the "intersect" user feature not doxxing people reading them.

I mean if that's the case a person not wanting other people to know they're reading something probably shouldn't put that on a tool designed to share what you're reading for other people (Goodreads).

I don't think anybody needs to be protected from themselves sharing things intentionally.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#235

Post by wtf242 »

That's super cool. I launched a book recommendations feature this year, which works vastly different. I ask users what their favorite books are(which you can rank), and then allow them to import their goodreads data which includes star reviews, and books they have read, then I determine your favorite style of books based on genres and subjects, then use opensearch to find similar books. It's a lot more complicated than that, but seems to work well. I'm always looking for ideas on how to improve this feature. Interested in what you are actually doing on the backend on the how-it-works page. thanks!

here's my recommendations feature: https://thegreatestbooks.org/recommendations

It's much more powerful if you're a member. you can restrict the results to certain genres or book lengths, as we as published date ranges, etc. If someone wants to try out the more powerful feature, DM me and i'll mark your account as a member.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#237

Post by caconym_ »

MichaelBosworth wrote: post #227
caconym_ wrote: post #190

Earlier quoted context omitted.

What does that comment have to do with confidentiality?

That he viewed a review on Goodreads as the reviewer’s intellectual property hadn’t occurred to me. I see why, in aggregate, many such opinions become valuable, but the whole is more than the sum of its parts. So does it feel to you guys like your comments, say, here in this Hacker News thread should be considered effectively copyrighted as your personal IP? If so, do you feel the same way about opinions you share ou…

There are well established legal standards for what is copyrightable and I believe written literary criticism trivially qualifies (as it should). Stuff you yell at the supermarket doesn't, IIUC, as it isn't fixed in a tangible form. Social media comments are, IIUC, generally protected. The exception would be comments that don't meet the bar to be considered "original", "creative", etc.

(not a lawyer)

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#238

Post by kulahan »

6stringmerc wrote: post #193
esskay wrote: post #135

Earlier quoted context omitted.

Fairly meaningless in this day and age. Also IIRC scraping legality depends heavily on jurisdiction. Some places take a more permissive view of accessing publicly available information, even if a site's TOS forbids bots. In the US there’s a major precedent [0] which held that scraping public-facing pages isn’t a CFAA "unauthorized access" issue. That’s a big part of why we’ve seen entire venture-backed scraping compa…

Tell that to judyrecords with the same smug attitude. Your textbook versus reality conceptualization of things is dogshit. It’s exploitation to do what OP did. You’re endorsing it and minimizing the ethics and this certainly shall poison the well from which you drink. Godspeed.

This is so overly dramatic it’s hard to even consider the point you’re trying to make.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#239

Post by irl_zebra »

voidUpdate wrote: post #124

Does this break part 4 of the Goodreads TOS? "[...] you agree not to sell, license, rent, modify, distribute, copy, reproduce, transmit, publicly display, publicly perform, publish, adapt, edit or create derivative works from any materials or content accessible on the Service. Use of the Goodreads Content or materials on the Service for any purpose not expressly permitted by this Agreement is strictly prohibited." Al…

This is, essentially, why I've withdrawn from posting content from my human brain almost anywhere on the open internet (except here, sometimes) and have retired blog posts, opinions, and so on to our friends WAN.

Re: Show HN: I scraped 3B Goodreads reviews to train a better recommendation model

#240

Post by loremm »

For intersect I also wonder if you add a filter that the books are within the top rated. Like if I give my favorite books and want to find someone who has my same taste, it doesn't help if they hated (all/most/some) of those books. Tricky in that not all users give star ratings
Post reply on HN