Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML is not that good at predicting consumers' choices

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

111–120 of 228 posts

Re: ML is not that good at predicting consumers' choices

#111

Post by Barrin92 »

teruakohatu wrote: post #27

> Currently, we are still far from a point where machines are able to abstract high-level concepts from data or engage in reasoning and reflection Of course when an AI does that, we then say its just doing statistics, not reasoning. Until you have built a recommendation engine from scratch, it is hard to appreciate the complexity. I don't mean the complexity of the code or algorithm (ALS and Spark are straightforward…

>Of course when an AI does that, we then say its just doing statistics, not reasoning.

no, AI simply doesn't do that. Even Demis Hassabis of Deepmind fame in a recent interview pointed this out. Machine learning is great on averaging out a large amount of data, which is often useful, but it doesn't generate true novelty in any human sense. AI can play Go, it can't invent Go.

In the same way today's recommender systems are great at averaging out my last 50 shopping items or spotify playlist but they can't take a real guess at what truly new thing I'd like based on a genuine understanding of say, my personality. Which is reflected in the quality of recommendations which is mostly "the thing you just bought/watched", which is ironically often incredibly uninteresting.

Re: ML is not that good at predicting consumers' choices

#112

Post by treesprite82 »

buscoquadnary wrote: post #38
aaronax wrote: post #34

ML is there to maximize business income--nothing else. If ML was benefiting me, it would know that 90% of the time I fire up Hulu I plan to watch the next episode of what I was watching last time. And it would make that a one click action. Instead I have to scroll past promotional garbage...every single time. Assholes.

Honestly a lot of this ML to me seems eerily similar to how in older times people would use sheep entrails or crow droppings to try and predict the future. I mean basically that is what ML is, trying to predict the future, the difference is they called it magic, we call it math, but both seem to have about the same outcome, or understandability.

> I mean basically that is what ML is, trying to predict the future

If being so reductive, that's also the scientific method. Form a model on some existing data, with the goal of it being predictive on new unseen data. Key is in favoring the more predictive models.

> they called it magic, we call it math, but both seem to have about the same outcome

Find me some sheep entrails that can do this: https://imagen.research.google/

Re: ML is not that good at predicting consumers' choices

#113

Post by ninkendo »

treesprite82 wrote: post #112
buscoquadnary wrote: post #38

Earlier quoted context omitted.

Honestly a lot of this ML to me seems eerily similar to how in older times people would use sheep entrails or crow droppings to try and predict the future. I mean basically that is what ML is, trying to predict the future, the difference is they called it magic, we call it math, but both seem to have about the same outcome, or understandability.

> I mean basically that is what ML is, trying to predict the future If being so reductive, that's also the scientific method. Form a model on some existing data, with the goal of it being predictive on new unseen data. Key is in favoring the more predictive models. > they called it magic, we call it math, but both seem to have about the same outcome Find me some sheep entrails that can do this: https://imagen.researc…

One of the oft overlooked, yet critically important aspects of the scientific method is the hypothesis. You don’t design an experiment having absolutely no idea what to expect. You have an educated guess in mind (the hypothesis), and you design the experiment in such a way that says “this result will rule out my hypothesis, while this other result might confirm it.”

Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.

Re: ML is not that good at predicting consumers' choices

#114

Post by bqmjjx0kac »

Eugr wrote: post #105
mbesto wrote: post #81

Earlier quoted context omitted.

I've always heard this, and so when I went for my first smart thermometer I went straight to Ecobee (which I'm very happy with btw). So I gotta ask HN...what the heck was so popular about Nests?! It's one thing to be go after shiny lures like new iPhone apps or luxury items...but a Thermostat?! Mind boggling...

It looks good on the wall, has a bright large display that lights up when you approach and intuitive enough for non-techies to operate. Also it can be installed without a common wire.

And you can control it from your phone, or with your voice by talking to Google assistant.

Re: ML is not that good at predicting consumers' choices

#115

Post by astrea »

bolasanibk wrote: post #55
ape4 wrote: post #22

You just bought a washing machine... could I interest you in a washing machine?

I cannot remember the reference now, but the reasoning I read was a person who just bought an item x might: 1. return the item if they are not satisfied with it and get a replacement Or 2. buy another one as a gift if they really like it. Both of these result in a higher fraction of conversions in this kind of targeting vs other targeting criteria.

I’m also more inclined to believe that the ad networks simply don’t have a way of knowing a sale was made.

Re: ML is not that good at predicting consumers' choices

#116

Post by switchbak »

cj wrote: post #3

While not exactly aligned to the research, I've been surprised how poor Nest Thermostat's learning feature is. The main selling point for Nest is having a "learning thermostat". Perhaps my schedule is just not predictable enough, but the auto-generated temperature schedules it generates after its "learning" period is not even close to what I would manually set up on a normal thermostat. Maybe I'm just an "edge case"…

The Ecobee is not really any better. It has various "features" which all end up in setting the temperature at a very uncomfortable setting even when you're home.

If I wanted those kind of savings, I could have just turned down my thermostat myself. Jeesh.

Matt Risinger (youtube expert builder guy) mentioned these are not anywhere near as valuable as they seem, and I'm inclined to agree. It's nice to be able to flick it on vacation mode when you're away I suppose.

I'd still buy it again, nice geeky metrics, and it's a quality company, but it doesn't save me anywhere close to 30% (or whatever the claim was).

Re: ML is not that good at predicting consumers' choices

#117

Post by Mordisquitos »

im3w1l wrote: post #36
ape4 wrote: post #22

You just bought a washing machine... could I interest you in a washing machine?

GPT can solve this! I prompted it with "Sarah bought a washing machine and a ". It completed "dryer.". Another "If you buy a hammer you might also want to buy " -> "a nail". Ill forgive the singular. Just to be clear those are not cherry picked - they were my first two attempts.

Fair enough regarding the washer/dryer, but I'm unimpressed by the hammer/nail example. If anything, I might want to buy "some nails" rather than "a nail", but I would argue that's not a brilliant guess either—if I'm buying a hammer I most likely already own the item that I want to hit with it. Some more interesting suggestions could have been for example "some pliers", "a set of screwdrivers", "a pair of work gloves", "safety goggles", etc.

Re: ML is not that good at predicting consumers' choices

#118

Post by jrumbut »

malkia wrote: post #75

Some years ago, I worked on a team "Ads Human Eval" - we had raters hired to do A/B testing for ads. These evaluated questionaires carefuly crafted by our linguists, and then analyzed by the statisticians providing feedback to the (internal) group that wanted to know more about. So the best experience was this internal event that we had, where the raters would say that certain Ad would not fare well (long term), whil…

> automation is not going to fully replace us

You say that the AI gave a bad answer, but it did give an answer right? Really fast? And it was cheaper than convening the panel of experts?

That's the fear of AI replacing humans. It's not that it works so well, it's that it works poorly (but fast and cheap).

Re: ML is not that good at predicting consumers' choices

#119

Post by jsjohnst »

ape4 wrote: post #22

You just bought a washing machine... could I interest you in a washing machine?

While this annoys the hell out of me too, there is a semi-logical explanation that makes sense in many cases:

The ad targeting knows you looked at washing machines, but doesn’t know you purchased.

Re: ML is not that good at predicting consumers' choices

#120

Post by linsomniac »

That's probably because people themselves are bad at predicting their own choices. Depending on mood, the day, even your hydration level, you can feel vastly different about something.

I believe this was a conclusion of the Netflix Prize contest, getting much more accurate at predictions was hard because people would not reliably rate a movie the same.

Post reply on HN