Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to recognize AI snake oil [pdf]

cs.princeton.edu

Post reply on HN
Print view View original

351–360 of 364 posts

Re: How to recognize AI snake oil [pdf]

#351

Post by SkyMarshal »

This is a good framework and categorization for evaluating the effectiveness of AI.

TLDR:

AI is good at: Perception (how things are)

AI is ok at: Judgement (how things are)

AI is poor at: Predicting (how things are going to be)

E.g., AI is effective to varying degrees at observing and characterizing how things are, but not at predicting how they're going to be.

For predicting, AI with hundreds of variables fairs no better than simple linear regression over single digits of variables. Simply because, the future is not reliably predictable, by any means.

Re: How to recognize AI snake oil [pdf]

#352

Post by dakial1 »

HlessClaudesman wrote: post #79

My brush with AI snake oil: I interviewed at a startup that seemed fishy. They offer a fully AI powered customer service chat as an off the shelf black box to banks. I highly suspect that they were a pseudo AI setup. LinkedIn shows that they are light on developers but very heavy on “trainers”, probably the people who actually handle the customers, mostly young graduates in unrelated fields, who may believe that thei…

Very common AI startup. They use some pre made AI tools (e.g. Watson bot) and resell it to specific industries where they already have the "intent trees" made (common questions/actions the user wants). The trainers are nothing more than analysts that will identify an intent not listed on the tree and configure it there. The devs probably are API and frontend devs, not much AI stuff going on.

Re: How to recognize AI snake oil [pdf]

#353

Post by m463 »

angstrom wrote: post #259
m463 wrote: post #254

Earlier quoted context omitted.

I think amazon works pretty well. They show me similar things to the thing I just put in my cart. Sometimes better choices that I made. They also show related things that other people have bought, that many times I end up purchasing.

Dirty secret: "The customers who bought also bought" algorithm doesn't require ML/AI. You can accomplish that with relational algebra on a precomputed data warehouse job and only for products with strong correlation. The intelligence of the customers is enough agency to instil a semblance of intelligence in the data.

I wonder if the dirty secret for lots of stuff is: "They aren't using AI anywhere"

Re: How to recognize AI snake oil [pdf]

#354

Post by swores »

Jestar342 wrote: post #296
andybak wrote: post #202

Earlier quoted context omitted.

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Amazon infuriates me. I regularly buy gadget-y bits like electronic components and peripherals. Probably at least once a month. Never see adverts for similar. That ONE TIME I buy a unicorn dress for my 2 year old daughter? That's a lifetime of unicorn related merchandise adverts and recommendations for you!

It's not impossible for that to be a clever decision by Amazon (although I'm not saying it's likely, I have no idea about the numbers).

The ultimate goal of the advertising is return on investment, not making you feel interested in the adverts. If, to exaggerate the possibility, 100% of "people who look at tech" are 0% influenced by adverts, but 10% of "people who bought a unicorn thing" will go on to buy another if they're constantly reminded that whoever they bought it for likes unicorns, all of a sudden it would make sense despite being counterintuitive to viewers.

A more commonly discussed example of a similar thing is that it's easy to think "I just bought a (dishwasher, keyboard, etc), I obviously already have one so why am I seeing adverts for them?" Sure, it might be that the company responsible has an incomplete profile and doesn't know you bought one already. But it's also possible that the % of people who just bought the item and then decide they don't like it, return it and buy a different type is high enough to be worth advertising to them.

Re: How to recognize AI snake oil [pdf]

#355

Post by deusofnull »

itronitron wrote: post #67
deusofnull wrote: post #61

Earlier quoted context omitted.

I worked for a company that had a "big data machine learning" product. Yeah, it was mostly elasticsearch aggregations.

well, aggregations are just leveraging how the data fields are analyzed and labeled, so if those were based on ML/NLP techniques then it could be legit

i believe the only NLP/ML stuff we really had was at the level of positive / negative sentiment analysis for social media posts, which as far as i know is pretty basic and mostly just involves dictionary checks for +/- words. the elasticsearch aggregations & indexing was not ML/NLP based at all.

Re: How to recognize AI snake oil [pdf]

#356

Post by MaxBarraclough »

mark-r wrote: post #144
wilg wrote: post #137

Earlier quoted context omitted.

Well... why is it necessary that we can teach a human to do something in order to teach a machine to do it?

Teaching a human is a heuristic for understanding the problem well enough to teach a machine.

This simply isn't true. We know your intuition here is mistaken, as we have plenty of counter-examples.

The best chess AIs can beat any human chess player. They use techniques that were never taught to them by a human.

Another example: a machine-learning-driven computer-vision system predicting the sex of a person based on an image of their iris. No human can do this. [0]

[0] Learning to predict gender from iris images (PDF) https://www3.nd.edu/~nchawla/papers/BTAS07.pdf

Re: How to recognize AI snake oil [pdf]

#357

Post by qnsi »

DenisM wrote: post #98
sgt101 wrote: post #94

Earlier quoted context omitted.

I guess the idea would be to get a few contracts, pull down some money from those and then go bust as the costs of the Mechanical Turk become evident?

> go bust as the costs of the Mechanical Turk become evident I'm afraid you have misspelled "raise a humongous round form SoftBank". It's an easy typo to make, don't feel bad.

Hugh mongous what?

Re: How to recognize AI snake oil [pdf]

#359

Post by angstrom »

m463 wrote: post #353
angstrom wrote: post #259

Earlier quoted context omitted.

Dirty secret: "The customers who bought also bought" algorithm doesn't require ML/AI. You can accomplish that with relational algebra on a precomputed data warehouse job and only for products with strong correlation. The intelligence of the customers is enough agency to instil a semblance of intelligence in the data.

I wonder if the dirty secret for lots of stuff is: "They aren't using AI anywhere"

Well, for problems that are simply looking for a ranked relationship if you have human input it could be used to train a ML that attempts to look for similar correlations...or you could just use your human mechanical turks that are already informing you. ML problems that are good are the ones not trying to approximate reality like CWBAB.

I'm doubtful we'll see an ai that makes a serious jump without directly interacting with the world we live in. Under that measure cars might be the closest since you learn to interact with the bounds of where it can and can't go is similar to a toddler learning to crawl.

Re: How to recognize AI snake oil [pdf]

#360

Post by jshen »

madrox wrote: post #140
reggieband wrote: post #58

I don't have time to read the entire paper but I would like to share an anecdote. I worked at a company with a well staffed/funded machine learning team. They were in charge of recommendation systems - think along the lines of youtube up next videos. My team wanted better recommendations (really, less editorial intensive) so the ML team spent weeks crafting 12 or more variants of their recommendation system for our c…

I had a similar experience at one point. A team put energy into building a recommendation system, and were able to demonstrate that the "Recommended for you" content performed better than all other content editorial. After getting challenged a bit, though, turns anything performs better when put under the header "Recommended for you."

You aren’t talking about dcom are you?
Post reply on HN