Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do recommender systems not seem good?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 68 posts

Re: Ask HN: Why do recommender systems not seem good?

#11

Post by behnamoh »

neovialogistics wrote: post #8

Scale. Providing accurate recommendation algorithms for thousands+++ of people across thousands+++ of data items is surprisingly expensive in compute and electricity. For any one user, sure you can do whatever you like. When you divide your resources across your userbase the prices get larger and larger.

Sam Altman said something about Instagram which is interesting. He said at midnight, the quality of the recommendations is higher because few people are on the platform.

Does it mean at peak hours IG switches to a simpler algorithm?

Otherwise, why would the quality of a RS drop during those hours?

Re: Ask HN: Why do recommender systems not seem good?

#12

Post by fl0id »

JohnMakin wrote: post #9

Because the recommendations aren't for you - they are what generates the most amount of ad revenue for them, which roughly correlates to the amount of money advertisers are willing to spend to reach people like you. That is in no way a guarantee that the recommendations will be good for you, because the incentives are not aligned.

This a lot. They aren’t optimizing for the user most likely.

Re: Ask HN: Why do recommender systems not seem good?

#13

Post by ozim »

andy99 wrote: post #2

They are not optimized for what you enjoy, or even want. A passable analogy: you buy a car and get hassled, often hard-sold for a pre-paid maintenance package, tire insurance, financing insurance, undercoating, bla bla. You don't want any of it but it's what they push the hardest.

That is biggest problem to explain to people that companies don’t do what’s best for their customers.

It is counter intuitive but yes companies do what is best for them. Often it also happens to be aligned but also quite often not.

Like loyal customers more often are ripped off because of vendor lock where new customers get huge discounts.

Re: Ask HN: Why do recommender systems not seem good?

#15

Post by JohnMakin »

ozim wrote: post #13
andy99 wrote: post #2

They are not optimized for what you enjoy, or even want. A passable analogy: you buy a car and get hassled, often hard-sold for a pre-paid maintenance package, tire insurance, financing insurance, undercoating, bla bla. You don't want any of it but it's what they push the hardest.

That is biggest problem to explain to people that companies don’t do what’s best for their customers. It is counter intuitive but yes companies do what is best for them. Often it also happens to be aligned but also quite often not. Like loyal customers more often are ripped off because of vendor lock where new customers get huge discounts.

> It is counterintuitive

Is it? This is how any (public) company works - they will do whatever it takes to make the shareholder return the highest. This inherently disregards what's best for the "customer." Any beliefs contrary to this is a naive belief that doing what the customer wants == the most revenue, e.g. "the customer is always right."

Maybe in some fields, but definitely not in circumstances like this where the "customer" is the product (or more accurately - the advertisers are the real customers).

By extension, "companies don't always do what's best for their customers" also applies to their employees, maybe even more so. I'm very certain that the largest US companies would kill their own employees if it was legal and resulted in the most cost savings/profit for them.

Re: Ask HN: Why do recommender systems not seem good?

#16

Post by [deleted] »

add-sub-mul-div wrote: post #4

I have a theory that some people's likes are based on things that are too subtle, intangible, or unrepeatable to identify, predict, or data mine. At least that's what I've figured is why recommendation systems don't work for me and why I run into so many dead ends when I try to use clear aspects of what I like to find other likes.

[deleted]

Re: Ask HN: Why do recommender systems not seem good?

#18

Post by beefnugs »

behnamoh wrote: post #7

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed. Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

Yes, absolutely hilarious, that after you buy a good air fryer, one that will last 5 years+ : These company's break every shred of moral and legal lines to hell and back to make sure that what happens is "WE HEARD YOU LOVE AIR FRYERS! here is 10 more!!!"

Re: Ask HN: Why do recommender systems not seem good?

#19

Post by Havoc »

behnamoh wrote: post #7

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed. Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

Was puzzled by this too, but realized that chances of a repeat buy are probably higher than a near blind shot at another category. So does make sense on some level

Re: Ask HN: Why do recommender systems not seem good?

#20

Post by SR2Z »

behnamoh wrote: post #11
neovialogistics wrote: post #8

Scale. Providing accurate recommendation algorithms for thousands+++ of people across thousands+++ of data items is surprisingly expensive in compute and electricity. For any one user, sure you can do whatever you like. When you divide your resources across your userbase the prices get larger and larger.

Sam Altman said something about Instagram which is interesting. He said at midnight, the quality of the recommendations is higher because few people are on the platform. Does it mean at peak hours IG switches to a simpler algorithm? Otherwise, why would the quality of a RS drop during those hours?

I bet their recommendation algorithms use some kind of iterative refinement to keep recommendations "fresh" and that at midnight it's just done with lower latency.
Post reply on HN