Live data from Hacker News

Ask HN: Why do recommender systems not seem good?

news.ycombinator.com

31–40 of 68 posts

Re: Ask HN: Why do recommender systems not seem good?

#31

Because the recommendations aren't for you - they are what generates the most amount of ad revenue for them, which roughly correlates to the amount of money advertisers are willing to spend to reach people like you. That is in no way a guarantee that the recommendations will be good for you, because the incentives are not aligned.

It's a tricky balance. The weaker the quality of recommendations, the more likely a user is to switch to another platform.

Re: Ask HN: Why do recommender systems not seem good?

#33
post #29

Mostly conjecture on my part, and an anecdote: for some things, I have very specific and niche tastes that come in the form of things that are difficult to label. Example: rap music and hiphop. For the most part, I don't enjoy it that much. There are a few things though that will make a track palatable to me (or instantly turn me off despite anything else positive about it): - Sentimentality or romance in the lyrics…

This was the promise of the Music Genome Project, the database for Pandora's recommendation engine.

Re: Ask HN: Why do recommender systems not seem good?

#34
post #22
post #7

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed. Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

“Why is Amazon trying to sell me an X? I just bought an X. Idiots!” Amazon is not an idiot: > You need two insights here: > 1) Conditional probability is a mathematical technology that does exist. > 2) Buying X is not entirely random across the population. > The ways X is not random vary based on the good [math]. Consider refrigerators. > You probably buy one every ten years. If I don't know where you are in your ref…

This sounds like something written by Nassim Taleb.

Re: Ask HN: Why do recommender systems not seem good?

#35
post #22
post #7

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed. Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

“Why is Amazon trying to sell me an X? I just bought an X. Idiots!” Amazon is not an idiot: > You need two insights here: > 1) Conditional probability is a mathematical technology that does exist. > 2) Buying X is not entirely random across the population. > The ways X is not random vary based on the good [math]. Consider refrigerators. > You probably buy one every ten years. If I don't know where you are in your ref…

That's kind of a dismissive way to say "maybe it's optimized for buyer's remorse or returns?"

Maybe it is, but I'm not totally convinced. For one it doesn't explain why Amazon would recommend literally the same SKU so often. Also, does Amazon really want to incentivize returning large items like fridges?

And the poker thing from the original post

> People who are compensated strictly based on their ability to predict the future, like poker players... tend to be much better at high school math than Twitter users.

Professional gamblers in games like poker usually don't try to predict the future. They just have a small set of hands with known probability and mainly have to focus on things like sizing their bets. (Poker players also have to read human signals, but I'm not sure they explicitly assign probabilities to these).

Another explanation is just that the cost of recommending you a fridge you already own is lower than the cost of tuning the algorithm better or having multiple algorithms depending on tuples of factors (price, customer, customer behavior). If that's true, then we should expect Amazon to do less of this in the future. If Patio11's explanation is correct, then we should expect Amazon to continue to recommend these items even as the algorithms are updated to the more recent generation of AI.

Re: Ask HN: Why do recommender systems not seem good?

#37
post #29

Mostly conjecture on my part, and an anecdote: for some things, I have very specific and niche tastes that come in the form of things that are difficult to label. Example: rap music and hiphop. For the most part, I don't enjoy it that much. There are a few things though that will make a track palatable to me (or instantly turn me off despite anything else positive about it): - Sentimentality or romance in the lyrics…

That was supposed to be the value proposition of Pandora and the music genome project. I don’t think the Pandora algorithm is very good, I’d guess they’re using something simpler.

Re: Ask HN: Why do recommender systems not seem good?

#38
post #30

Have you used the tiktok one? It's too good for my liking. It can make weird inferences that no other algo can (this is actually why I deleted tiktok, far too addictively good)

For non-tiktok users, what types of media is it useful as a recommendation engine?

Re: Ask HN: Why do recommender systems not seem good?

#39
post #14

Often they just use "people who watched X also watch Y", which actually selects for the most popular stuff instead of the most similar.

Most similar has some hilarious pitfalls too though. I saw a talk by someone who worked on the google play music, or whatever it's called these days, recommendation system and his canonical example was a pure similarity metric would recommend the all-women band "Lez Zepplin" to fans of Led Zeppelin. How's that going to score amongst listeners?
Post reply on HN