Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML is not that good at predicting consumers' choices

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

51–60 of 228 posts

Re: ML is not that good at predicting consumers' choices

#51

Post by fshbbdssbbgdd »

cj wrote: post #3

While not exactly aligned to the research, I've been surprised how poor Nest Thermostat's learning feature is. The main selling point for Nest is having a "learning thermostat". Perhaps my schedule is just not predictable enough, but the auto-generated temperature schedules it generates after its "learning" period is not even close to what I would manually set up on a normal thermostat. Maybe I'm just an "edge case"…

Not only does the Nest ignore my preferences, I think it actually lies about the current temperature.

Example:

Setting is 72, reading is 73. AC is not on, I guess the thermostat is trying to save energy. I lower setting to 71, reading instantly drops to 72! I don’t think it’s a coincidence, this has happened several times.

Re: ML is not that good at predicting consumers' choices

#52

Post by mrguyorama »

abirch wrote: post #50
mgraczyk wrote: post #42

Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x. I…

Amazon does a remarkably good job of predicting what I'll buy and I frequently add to my purchases.

Are you the mythical person buying 15 vacuum cleaners at the same time?

Re: ML is not that good at predicting consumers' choices

#53

Post by mrxd »

mgraczyk wrote: post #42

Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x. I…

If your ML model is able to predict what consumers are going to buy, the revenue lift would be zero.

Let's say I go to the store to buy milk. The store has a perfect ML model, so they're able to predict that I'm about to do that. I walk into the store and buy the milk as planned. So how does the ML help drive revenue? The store could make my life easier by having it ready for me at the door, but I was going to buy it anyway, so the extra work just makes the store less profitable.

Maybe they know I'm driving to a different store, so they could send me an ad telling me to come to their store instead. But I'm already on my way, so I'll probably just keep going.

Revenue comes from changing consumer behavior, not predicting it. The ideal ML model would identify people who need milk, and predict that they won't buy it.

Re: ML is not that good at predicting consumers' choices

#54

Post by abirch »

mrguyorama wrote: post #52
abirch wrote: post #50

Earlier quoted context omitted.

Amazon does a remarkably good job of predicting what I'll buy and I frequently add to my purchases.

Are you the mythical person buying 15 vacuum cleaners at the same time?

No, I'm the person who doesn't know the great things to buy with my Raspberry Pi. Thanks to great predictions from Amazon's part, they get me to buy more. Similar to how Netflix does a pretty good job of recommending movies.

Re: ML is not that good at predicting consumers' choices

#55

Post by bolasanibk »

ape4 wrote: post #22

You just bought a washing machine... could I interest you in a washing machine?

I cannot remember the reference now, but the reasoning I read was a person who just bought an item x might: 1. return the item if they are not satisfied with it and get a replacement Or 2. buy another one as a gift if they really like it.

Both of these result in a higher fraction of conversions in this kind of targeting vs other targeting criteria.

Re: ML is not that good at predicting consumers' choices

#56

Post by qvrjuec »

mrxd wrote: post #53
mgraczyk wrote: post #42

Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x. I…

If your ML model is able to predict what consumers are going to buy, the revenue lift would be zero. Let's say I go to the store to buy milk. The store has a perfect ML model, so they're able to predict that I'm about to do that. I walk into the store and buy the milk as planned. So how does the ML help drive revenue? The store could make my life easier by having it ready for me at the door, but I was going to buy it…

If the store knows you will want to buy milk, it will have milk in stock according to demand. If it doesn't have a perfect understanding of whether or not people want to buy milk, the store will over/under stock and lose money.

Re: ML is not that good at predicting consumers' choices

#57

Post by soared »

mrxd wrote: post #53
mgraczyk wrote: post #42

Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x. I…

If your ML model is able to predict what consumers are going to buy, the revenue lift would be zero. Let's say I go to the store to buy milk. The store has a perfect ML model, so they're able to predict that I'm about to do that. I walk into the store and buy the milk as planned. So how does the ML help drive revenue? The store could make my life easier by having it ready for me at the door, but I was going to buy it…

This is incorrect. You can predict many things that drive incremental revenue lift.

The simplest: Predict what features a user is most interested in, drive them to that page (increasing their predicted conversion rate) -> purchases that occur now that would not have occurred before.

Similarly: Predict products a user is likely to purchase given they made a different purchase. The user may not have seen these incremental products. For example, users buys orange couch, show them brown pillows.

Like above, the same actually works for entirely unrelated product views. If users views x,y,z products we can predict they will be interested in product w and we can advertise it.

Or we predict a user was very likely to have made a purchase, but hasn’t yet. Then we can take action to advertise to them (or not advertise to them).

Re: ML is not that good at predicting consumers' choices

#58

Post by mgraczyk »

nojito wrote: post #48
mgraczyk wrote: post #42

Always interesting to see outsiders writing papers about this, using anecdote and unrelated data (mostly political and real world purchase data in this case) to argue that ML doesn't make useful predictions. Meanwhile I look at randomized controlled trial data showing millions of dollars in revenue uplift directly attributable to ML vs non-ML backed conversion pipelines, offsetting the cost of doing the ML by >10x. I…

>Always interesting to see outsiders writing papers about this I don't think you know who andrew gelman is. Additionally, that's not the conclusion derived from this study.

The actual conclusion of the study is so absurd that it's not worth engaging with seriously.

    That is, to maximally understand, and therefore predict, consumer preferences is likely to require information outside of data on choices and behavior, but also on what it is like to be human.
I was responding to the interpretation from the blog post, which is more reasonable.

Re: ML is not that good at predicting consumers' choices

#60

Post by idontpost »

dominotw wrote: post #35
Slackwise wrote: post #10

Earlier quoted context omitted.

"Why am I sweating right now? Oh, the Nest set the temperature too high again!" And then after a few instances, I just turn off all the automation and set up a schedule like normal. Same with the "away from home" which seems to randomly think I'm away and I have no idea why. Oh, and the app doesn't show me filter reminders, only the actual device, which I never touch all the way downstairs. There's not even any statu…

Maybe it considers environmental impact of air conditioning in its models and tries to nudge users into tolerating higher temps.

If you have to guess why it's making decisions you don't want, it's a shitty product.
Post reply on HN