Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to recognize AI snake oil [pdf]

cs.princeton.edu

Post reply on HN
Print view View original

281–290 of 364 posts

Re: How to recognize AI snake oil [pdf]

#281

Post by visarga »

chimi wrote: post #176
mark-r wrote: post #144

Earlier quoted context omitted.

Teaching a human is a heuristic for understanding the problem well enough to teach a machine.

I agree and rather than post a sibling response, I'll add that I think it's necessary today , simply because we don't have AGI, yet . And also point out that we are talking about determining if AI is snake oil or not. There may be some scenarios where we can teach a computer to do something we can't teach a human to do, I can't think of any off the top of my head, but if we can't, then I'm going to be super doubtful…

Same for AlphaGo, but it only works when you have access to cheap simulation (breakout being a game easy to run, Go being just a board). It doesn't work as well in situations where you don't see the full state of the system, or where there is randomness.

Re: How to recognize AI snake oil [pdf]

#282

Post by _blu »

nradov wrote: post #227
Enginerrrd wrote: post #190

Earlier quoted context omitted.

Right? Recommenders are almost counter productive for me in most cases. I want a recommender to remain broad, not give me increasingly niche recommendations a la youtube. About the only halfway decent recommender system I've interacted with is the various forms of curated lists on Spotify. They seem to actually take a decent stab at it with related but sufficiently different and interesting content. For all Facebook…

Likewise Pandora does a decent job of picking songs for me based on what I previously liked. It's not perfect but far better than random.

it's easier to recomment things you probably like than things they want you to like/may like

Re: How to recognize AI snake oil [pdf]

#283

Post by Tomte »

ineedasername wrote: post #232
stefan_ wrote: post #181

Earlier quoted context omitted.

Since we are sharing anecdotes, I can report it's been 20 years of buying stuff on the internet and the combined billions of ad tracking research dollars spent by Amazon and Google have not yet come up with a better algorithm than to bombard me with ads for the exact same thing I just bought .

Sometimes it also recommends other brands of the thing I just bought. Like a cell phone: I didn't buy Samsung, but now I see their ads. I guess that is a very very small improvement- Maybe I'm the type of consumer who gets a new phone every week?

No, but you might be unhappy with your phone, return it and buy a new one.

Re: How to recognize AI snake oil [pdf]

#284

Post by DonHopkins »

The classic Bone Tumor Differential Diagnosis program that Apple published for the Apple ][ in the original Apple Software Bank Volume 1 from 1978 sounds legit. But beware: it requires 32K of RAM, bigger than any other program in that volume! (Available on both cassette and 5-1/4" floppy disk for free from your local Apple dealer.)

https://archive.org/stream/Apple_Software_Bank_Vol_1-2/Apple...

https://archive.org/details/a2_Biology_19xx_

https://mirrors.apple2.org.za/ftp.apple.asimov.net/documenta...

Program Name: BONE TUMOR DIFFERENTIAL DIAGNOSIS

Software Bank Number: 001 1 4

Submitted By: Jeffrey Dach, M.D.

Program Language: APPLESOFT II BASIC

Minimum Memory Size: 32K Bytes

This program is intended for use by qualified medical practitioners. While the specific data are of interest only to those familiar with bone pathologies, the programming techniques may well interest a wide range of computer users.

INSTRUCTIONS

LOAD the program into APPLESOFT II BASIC, and type RUN. Follow the instructions displayed on the screen. The program asks a series of questions concerning radiographic and clinical details of the bone tumor in question. For each question, type the number of the appropriate answer and press the RETURN key. Finally, the program uses Baye's rule and a predetermined probability matrix from Lodwick (1963) to calculate the relative probabilities of 9 different diagnoses.

Some knowledge of descriptive terms for bone tumors is needed to answer the questions. Only a qualified physician should attempt to use this program as a diagnostic tool.

Re: How to recognize AI snake oil [pdf]

#285

Post by mojuba »

Enginerrrd wrote: post #190
stefan_ wrote: post #181

Earlier quoted context omitted.

Since we are sharing anecdotes, I can report it's been 20 years of buying stuff on the internet and the combined billions of ad tracking research dollars spent by Amazon and Google have not yet come up with a better algorithm than to bombard me with ads for the exact same thing I just bought .

Right? Recommenders are almost counter productive for me in most cases. I want a recommender to remain broad, not give me increasingly niche recommendations a la youtube. About the only halfway decent recommender system I've interacted with is the various forms of curated lists on Spotify. They seem to actually take a decent stab at it with related but sufficiently different and interesting content. For all Facebook…

> For all Facebook knows about me they've always been exceptionally bad at advertising to me

Because it's not Facebook really, it's the advertisers who choose targeting criteria. You as an advertiser have a myriad of options. For example if you've built a competitor to X, you can target users who've visited X recently, aged N-M, residing in countries A, B and C, and so on. There are options with broader interests too. Poorly targeted ad means poorly selected criteria by the advertiser (or sometimes just advertiser experimenting) and consequently money wasted. Facebook doesn't care though.

Then there is retargeting/remarketing (target bounced traffic) already mentioned here is probably the stupidest looking invention that actually works.

Re: How to recognize AI snake oil [pdf]

#286

Post by DonHopkins »

gok wrote: post #56

I really wish we could stop using AI or ML for things in the "predicting social outcomes" category. Naming them more like "computational astrology" or "machine alchemy" would be a better fit.

Apple was all over that in 1979.

https://i.imgur.com/ssFqU3i.jpg

Re: How to recognize AI snake oil [pdf]

#287

Post by jusonchan81 »

I once figured I can use ML to predict stock market movements and I went about buying 5 years of data from CBOE and started working on modeling. At one point I managed to get fairly accurate predictions of movements and I thought I had struck a lottery.

Turns out my training data had tomorrow’s movement indicator in it which I accidentally added. Despite that dumb mistake it was only accurate up to 80% or so. That’s when I realized Im better off with a random number generator.

Re: How to recognize AI snake oil [pdf]

#288

Post by smcl »

andybak wrote: post #202
stefan_ wrote: post #181

Earlier quoted context omitted.

Since we are sharing anecdotes, I can report it's been 20 years of buying stuff on the internet and the combined billions of ad tracking research dollars spent by Amazon and Google have not yet come up with a better algorithm than to bombard me with ads for the exact same thing I just bought .

I just spent 15m on Amazon trying to prod the recommendation algorithm into finding something I actually wanted to buy so I could get above the "free delivery threshold". Think about that. I wanted to spend money. I wasn't too fussy what it was. Amazon has a decade of my purchasing and browsing history. And they still failed.

Amazon still suffers from those "This guy really loves washing machines!" type recommendations

Re: How to recognize AI snake oil [pdf]

#290

Post by dcminter »

iamsb wrote: post #240
btilly wrote: post #129

Earlier quoted context omitted.

This reminds me of a job interview I was on. I was asked about how I would use AI/machine learning for their problem space. Since they seemed to be smart and level-headed, I answered honestly, "Pick something unimportant, use a machine learning algorithm just to get familiar with the tools, ignore the result unless it happens to work, then put machine learning in your marketing materials. But keep track of it, and if…

Speaking of jobs and interviews. I am yet to find a job board which does not show JavaScript jobs when searching for Java jobs. Some of them claim to use AI. :)

In fairness I have this issue with a number of human intelligence based (allegedly) recruiters.
Post reply on HN