Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to recognize AI snake oil [pdf]

cs.princeton.edu

Post reply on HN
Print view View original

291–300 of 364 posts

Re: How to recognize AI snake oil [pdf]

#291

Post by f137 »

andybak wrote: post #202
stefan_ wrote: post #181

Earlier quoted context omitted.

Since we are sharing anecdotes, I can report it's been 20 years of buying stuff on the internet and the combined billions of ad tracking research dollars spent by Amazon and Google have not yet come up with a better algorithm than to bombard me with ads for the exact same thing I just bought .

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

I hope 'm' here is minutes, not millions )

Re: How to recognize AI snake oil [pdf]

#292

Post by rusticpenn »

visarga wrote: post #280
chimi wrote: post #165

Earlier quoted context omitted.

You're touching on the "difficulty" in verbalizing it. I see what you mean, because you did learn that the heuristic was changing with just a yes or no. I said you can't teach that way, but you clearly learned that way, so I wasn't exactly correct, but I'm not practically wrong either still I don't think. I wonder, how would an AI perform on the same test. What is the mathematical minimum number of questions on such…

> I wonder, how would an AI perform on the same test. This situation is called Multi-armed Bandit. In this setup you have a number of actions at your disposal and need to maximise rewards by selecting the most efficient actions. But the results are stochastic and the player doesn't know which action is best. They need to 'spend' some time trying out various actions but then focus on those that work better. In a varia…

If the rewards are changing, then isnt it a moving target problem?

Re: How to recognize AI snake oil [pdf]

#293

Post by ppf »

iamsb wrote: post #240
btilly wrote: post #129

Earlier quoted context omitted.

This reminds me of a job interview I was on. I was asked about how I would use AI/machine learning for their problem space. Since they seemed to be smart and level-headed, I answered honestly, "Pick something unimportant, use a machine learning algorithm just to get familiar with the tools, ignore the result unless it happens to work, then put machine learning in your marketing materials. But keep track of it, and if…

Speaking of jobs and interviews. I am yet to find a job board which does not show JavaScript jobs when searching for Java jobs. Some of them claim to use AI. :)

My wife once watched me as I did a job search, carefully setting all the available parameters to match my requirements. She said "don't forget, engineering jobs are posted by admin staff", typed "software engineer location" into Google, and found way more than I ever did.

Re: How to recognize AI snake oil [pdf]

#294

Post by sgt101 »

erikig wrote: post #158
sgt101 wrote: post #87

I read "Why are HR departments apparently so gullible?" and as someone who has worked in a corporate for 20 years I spotted my underwear. The identification of facial recognition as problematic because of accuracy doesn't match my thinking. I believe that the key issue is that given a set of targets facial recognition systems will find near misses from the wider population of all faces offered as candidates, that the…

...I spotted my underwear? What does this mean?

English humour - being taken unaware by a sudden very funny thought causing one to involuntarily do a small wee in your pants. This should not be taken as a literal admission of incontinence.

Re: How to recognize AI snake oil [pdf]

#295

Post by classified »

sriku wrote: post #273
HlessClaudesman wrote: post #79

My brush with AI snake oil: I interviewed at a startup that seemed fishy. They offer a fully AI powered customer service chat as an off the shelf black box to banks. I highly suspect that they were a pseudo AI setup. LinkedIn shows that they are light on developers but very heavy on “trainers”, probably the people who actually handle the customers, mostly young graduates in unrelated fields, who may believe that thei…

I don't think that is in principle problematic (unlike the social problem statements pointed out in the talk). A system which amplifies human resources by filling in for their common activities over time could use sophisticated tech drawing on the latest in NLP. The metric would be a ratio of the number of service requests they handle per day / the number of "trainers" (or whatever name given) compared to the median…

My experience with automated "help" desks is that I have to let the automatons fail one after the other until I finally get connected to a real human. Then I can start to state my problem. All that those automated substitutes do is discourage customers from calling at all.

Re: How to recognize AI snake oil [pdf]

#296

Post by Jestar342 »

andybak wrote: post #202
stefan_ wrote: post #181

Earlier quoted context omitted.

Since we are sharing anecdotes, I can report it's been 20 years of buying stuff on the internet and the combined billions of ad tracking research dollars spent by Amazon and Google have not yet come up with a better algorithm than to bombard me with ads for the exact same thing I just bought .

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Amazon infuriates me. I regularly buy gadget-y bits like electronic components and peripherals. Probably at least once a month. Never see adverts for similar.

That ONE TIME I buy a unicorn dress for my 2 year old daughter? That's a lifetime of unicorn related merchandise adverts and recommendations for you!

Re: How to recognize AI snake oil [pdf]

#297

Post by mcdevilkiller »

ffe44 wrote: post #239
andybak wrote: post #202

Earlier quoted context omitted.

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Ha. Last time I wanted to buy something on Amazon, their search page kept freezing every time I load it: 100% CPU load. Because I really wanted to buy that thing, I spent 30 mins debugging their silly scripts and found one for-loop that tries to find a non-existent element. Unfortunately, I couldn't figure out a way to enable my fix in a minified script, as reloading the page kept loading the original script.

Whit uMatrix, you could have created a rule to block that script.

Re: How to recognize AI snake oil [pdf]

#298

Post by edoloughlin »

ffe44 wrote: post #239
andybak wrote: post #202

Earlier quoted context omitted.

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Ha. Last time I wanted to buy something on Amazon, their search page kept freezing every time I load it: 100% CPU load. Because I really wanted to buy that thing, I spent 30 mins debugging their silly scripts and found one for-loop that tries to find a non-existent element. Unfortunately, I couldn't figure out a way to enable my fix in a minified script, as reloading the page kept loading the original script.

Put a breakpoint on the offending line, manually add the element when it breaks, hit resume and hope for the best?

Re: How to recognize AI snake oil [pdf]

#299

Post by classified »

codeulike wrote: post #3

Perhaps ML could be applied here to help filter out the barrage of AI snake oil. Funding, anyone?

That filter would be ludicrously easy to "train". Just grep(1) for the letters 'AI'. I'll blithely take millions and billions of funding for this brilliantly profound vision of a mission to improve life for every single human being on earth for all time to come.

Re: How to recognize AI snake oil [pdf]

#300

Post by radarsat1 »

Jestar342 wrote: post #296
andybak wrote: post #202

Earlier quoted context omitted.

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Amazon infuriates me. I regularly buy gadget-y bits like electronic components and peripherals. Probably at least once a month. Never see adverts for similar. That ONE TIME I buy a unicorn dress for my 2 year old daughter? That's a lifetime of unicorn related merchandise adverts and recommendations for you!

Actually that's really interesting because it exposes a bias in their recommendation system. They must be heavily biased towards things with mass appeal instead of specifically targeting user preferences, which is funny because it goes against the grain of the whole "targeted advertising" promise of ML. You'd think if anyone could get that right, it would be Amazon, yet..
Post reply on HN