Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML is not that good at predicting consumers' choices

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

121–130 of 228 posts

Re: ML is not that good at predicting consumers' choices

#121

Post by teruakohatu »

Barrin92 wrote: post #111
teruakohatu wrote: post #27

> Currently, we are still far from a point where machines are able to abstract high-level concepts from data or engage in reasoning and reflection Of course when an AI does that, we then say its just doing statistics, not reasoning. Until you have built a recommendation engine from scratch, it is hard to appreciate the complexity. I don't mean the complexity of the code or algorithm (ALS and Spark are straightforward…

>Of course when an AI does that, we then say its just doing statistics, not reasoning. no, AI simply doesn't do that. Even Demis Hassabis of Deepmind fame in a recent interview pointed this out. Machine learning is great on averaging out a large amount of data, which is often useful, but it doesn't generate true novelty in any human sense. AI can play Go, it can't invent Go. In the same way today's recommender system…

> but it doesn't generate true novelty in any human sense.

Few humans do either, even great artists create from, or react to, the art they experienced in their life. At the end of the day this is a rather metaphysical question. A Go model is trained to play Go, not create new games.

> on a genuine understanding of say, my personality

That is exactly what they are trying to do. The purpose of many recommendation systems is to uncover the latent variables and categories that you might like. They are not just averaging what you listened to. It is feasible (but very unwise) to predict certain users like to listen to happy upbeat tracks on Fridays, and sad songs sung in Romance languages on Mondays.

Each time you skip a track or relisten to a track (implicit feedback) or like a track (explicit feedback) you are giving it information, which might be more than just the interaction (the timestamp, the IP address location, how quick you were to skip the track etc.).

On the other hand, the purpose of those systems is simply to optimize what generates the most short term income, in which case, you get 'boring'/simple shopping recommendations that on average make more profit.

Systems have tried to predict model the human experience, but in the past there have been epic fails so nobody is trying to explicitly do that anymore. The classic example would be stores deciding teens were pregnant and inadvertently alerting their parents. Nobody wants to see ads for Amazon's in-house toilet paper brand just as they are about to get up to go to the toilet.

Re: ML is not that good at predicting consumers' choices

#122

Post by jsjohnst »

thaumasiotes wrote: post #102
armchairhacker wrote: post #47

Earlier quoted context omitted.

I think the reason this happens is that when you start looking for washing machines, you start getting ads for them. Then when you buy nobody tells the ad companies that you just bought a washing machine so they still send you ads because they think you’re still looking. Even if you just went straight to the model site and clicked “buy”.

We know that's not the reason; Amazon is infamous for advertising washing machines to people who have just bought a washing machine from Amazon.

Amazon can’t suggest a “product you might like” if their life depended on it. They have 19 years of purchase history for me spanning thousands of product categories. Want to know a category I’ve never looked at or bought from Amazon, women's handbags. What’s near the top of the suggestions right now on the homepage for me? You guessed it!

Re: ML is not that good at predicting consumers' choices

#123

Post by Aerroon »

thaumasiotes wrote: post #101
Aerroon wrote: post #95

Earlier quoted context omitted.

Part of the reason they're horrible is because people don't have consistent interests. I might be interested in raunchy content right now, but I won't be a few hours later. What determines whether I'm interested in the former is outside of the control of these algorithms - they don't know all of the external events that can change my current mood and preferences. As a result of this it makes sense for people to have…

> Part of the reason they're horrible is because people don't have consistent interests. I might be interested in raunchy content right now, but I won't be a few hours later. What determines whether I'm interested in the former is outside of the control of these algorithms I don't think that matters at all. People don't complain that they're getting recommendations that would have been great if they had come in an ho…

I think it matters a lot, because that "awful content" is popular enough that others watch it. People can watch and read things they swear they would never be interested in, but this only happens sometimes. The algorithm taking cues from that is itself a discouraging factor on clicking on them.

Re: ML is not that good at predicting consumers' choices

#124

Post by zach_garwood »

wrycoder wrote: post #98
tomcam wrote: post #29

I can personally vouch that Amazon, Twitter, and YouTube all do horrible horrible jobs predicting my taste. And they have got worse over the years, not better

I think YouTube has given up on figuring me out. They mostly offer stuff I’ve already watched or stuff on my watch list.

Same. For a long time they wanted me to watch angry white guys complaining about pop culture. But all I want to watch are PBS science shows and stuff about ancient history! Eventually, they gave up, and now half the videos they recommend are ones I already watched years ago.

Re: ML is not that good at predicting consumers' choices

#126

Post by Asooka »

ape4 wrote: post #22

You just bought a washing machine... could I interest you in a washing machine?

A slightly more ridiculous one I experienced recently was when I searched where to buy tool X. Started getting ads for why I need tool X and why it's the best tool ever. I already want one, I'm looking where to buy, not trying to learn what it's for!

Re: ML is not that good at predicting consumers' choices

#127

Post by malkia »

jrumbut wrote: post #118
malkia wrote: post #75

Some years ago, I worked on a team "Ads Human Eval" - we had raters hired to do A/B testing for ads. These evaluated questionaires carefuly crafted by our linguists, and then analyzed by the statisticians providing feedback to the (internal) group that wanted to know more about. So the best experience was this internal event that we had, where the raters would say that certain Ad would not fare well (long term), whil…

> automation is not going to fully replace us You say that the AI gave a bad answer, but it did give an answer right? Really fast? And it was cheaper than convening the panel of experts? That's the fear of AI replacing humans. It's not that it works so well, it's that it works poorly (but fast and cheap).

Good point, and there were several remarkable cases in the past years. Also it really depends with what source that AI was trained.

Re: ML is not that good at predicting consumers' choices

#128

Post by Melatonic »

Slackwise wrote: post #10
cj wrote: post #3

While not exactly aligned to the research, I've been surprised how poor Nest Thermostat's learning feature is. The main selling point for Nest is having a "learning thermostat". Perhaps my schedule is just not predictable enough, but the auto-generated temperature schedules it generates after its "learning" period is not even close to what I would manually set up on a normal thermostat. Maybe I'm just an "edge case"…

"Why am I sweating right now? Oh, the Nest set the temperature too high again!" And then after a few instances, I just turn off all the automation and set up a schedule like normal. Same with the "away from home" which seems to randomly think I'm away and I have no idea why. Oh, and the app doesn't show me filter reminders, only the actual device, which I never touch all the way downstairs. There's not even any statu…

The problem I think with all of these ML functions is that there is never enough of an in between full manual and full auto magic. The Nest could simply ask you a few times a day how you feel about how its doing (too hot ? too cold? too expensive?) or you could opt in somewhere and to give feedback. And then it could keep doing its behind the scenes mumbo jumbo.

Re: ML is not that good at predicting consumers' choices

#129

Post by Melatonic »

kayodelycaon wrote: post #16
cj wrote: post #3

While not exactly aligned to the research, I've been surprised how poor Nest Thermostat's learning feature is. The main selling point for Nest is having a "learning thermostat". Perhaps my schedule is just not predictable enough, but the auto-generated temperature schedules it generates after its "learning" period is not even close to what I would manually set up on a normal thermostat. Maybe I'm just an "edge case"…

Things like this are exactly why I went with less "intelligent" smart thermostat. (Honeywell T9) The only learning feature it has is figuring out how long it takes to heat or cool the house given the current weather. Before a schedule change, can heat or cool the house so it hits next target temperature on time. This seems to work extremely well. Everything else like schedule and away settings are configured by the u…

Sounds nice. Let the machine figure out something other than people - it will probably be much better at that.

Re: ML is not that good at predicting consumers' choices

#130

Post by malkia »

Melatonic wrote: post #97
malkia wrote: post #75

Some years ago, I worked on a team "Ads Human Eval" - we had raters hired to do A/B testing for ads. These evaluated questionaires carefuly crafted by our linguists, and then analyzed by the statisticians providing feedback to the (internal) group that wanted to know more about. So the best experience was this internal event that we had, where the raters would say that certain Ad would not fare well (long term), whil…

I dont think most people are arguing that machines will replace everyone anytime soon - it is that they will replace a huge portion of people. If one person can do the job of 10,000 by being the tweaker / approver of an advanced AI that is still 9,999 jobs eliminated. That might be hyperbole (you still probably will need people to support that system)

I agree, but it's true that some jobs should not simply exists.

To this day, if I go to our airport in Sofia (Bulgaria), and my baggage is over the limit of 20 or was it 25kg I have to go to another place, pay for it and come back (why? bureaucracy - not only I have to do it, but I'm slowing anyone waiting for me to this - it's like 25-50 meters one place to the other)

Unlike Frakfurt, Munich or Heathrow airport where I can pay that fine right there.

Some jobs simply should not exist.

Post reply on HN