Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is artificial intelligence permanently inscrutable?

nautil.us

Post reply on HN
Print view View original

31–40 of 69 posts

Re: Is artificial intelligence permanently inscrutable?

#31

Post by venning »

The pneumonia-asthma example seems to be an example of a Simpson's paradox [1]. The doctors acted on a strong (accurate) belief about asthma sufferers contracting pneumonia and acted in such a way that the data obscured an actual causal link (asthma as an aggravating factor to pneumonia). This is opposed to the canonical Simpson's paradox where doctors acted on a strong (inaccurate) belief about severe kidney stones [1a] and again produced lopsided data that hid the best treatment option until the paradox was identified.

Humans have a very hard time uncovering so-called "lurking variables" [2] and identifying such paradoxes. I don't see how a neural network (or other machine learning tool) could do so on their own, but I don't know that much about machine learning. So, I guess I have two questions for the experts out there:

* If all training data is affected by a confounding variable, can a machine learning algorithm identify its existence, or is it limited by only knowing a tainted world?

* Once we have identified such lopsided data and understood its cause, how do you feed that back into your algorithm to correct for it?

---

[1] https://en.wikipedia.org/wiki/Simpson%27s_paradox

[1a] https://en.wikipedia.org/wiki/Simpson%27s_paradox#Kidney_sto...

[2] https://en.wikipedia.org/wiki/Confounding

Re: Is artificial intelligence permanently inscrutable?

#32

Post by vertex-four »

cscurmudgeon wrote: post #21
peterbonney wrote: post #18

Earlier quoted context omitted.

Or: nobody understands exactly how your doctor's brain reaches its conclusion about what's wrong with you, but you still trust it.

It is not like the doctor can explain in English when you ask him for details. Not all.

Or, to put it in other words - how your car's AI decides that there's five cars around you probably isn't that interesting. How it decides that based on its input, it should probably slam the brakes, is interesting.

Re: Is artificial intelligence permanently inscrutable?

#33

Post by Inlinked »

dave_sullivan wrote: post #14
yoav_hollander wrote: post #2

I think this issue of "Explainable Machine Learning" and interpretability is just going to get more and more important as ML grows. It will also be important for verifying ML-based systems - another problem area. See [1] for a discussion of both. [1] https://blog.foretellix.com/2016/08/31/machine-learning-veri...

I really disagree and think the whole "There's no way to gauge results!" meme is low-impact FUD. FUD that isn't particularly dangerous so people that don't know any better just believe it (like python can't be performant because of the GIL or macs are better than PCs or some other inanity). > As exciting as their performance gains have been, though, there’s a troubling fact about modern neural networks: Nobody knows…

There is no strong theoretical foundation for very complex and/or deep models. There are ways to gauge results, but right now, the media is right in that there are risks.

The pneumonia project referenced in the article had Caruana voting against implementing the most accurate model: A neural net. Instead they went with a way less accurate logistic regression model, one they could safely implement in production, inspect, explain, and defend to the doctors.

Nobody does quite know why neural networks work so well. There is a Nobel Prize waiting there for someone or some team to solve this with mathematical (or physics) rigor.

Nodes in neural network layers can represent multiple features, or share feature representations. Do we know if a neural net (and which part) is targeting skin color, or acne? Do we know that credit risk models are targeting sex, even though we left out this feature (it may infer this from other features)?. Depending on the application, this is important to know for certain.

Reverse engineering sure does work, but can we fully find out the source code from a program, just by fuzzing inputs and looking at outputs? Or are we only looking at (perhaps a small part of) its behavior?

> You will run into trouble if you don't know what you're doing

Likewise: You will run into trouble, if you don't know for sure what your models are doing.

http://blogs.wsj.com/digits/2015/07/01/google-mistakenly-tag...

> as per common sense

https://en.wikipedia.org/wiki/Commonsense_reasoning#Commonse...

Re: Is artificial intelligence permanently inscrutable?

#35

Post by [deleted] »

cscurmudgeon wrote: post #21
peterbonney wrote: post #18

Earlier quoted context omitted.

Or: nobody understands exactly how your doctor's brain reaches its conclusion about what's wrong with you, but you still trust it.

It is not like the doctor can explain in English when you ask him for details. Not all.

[deleted]

Re: Is artificial intelligence permanently inscrutable?

#36

Post by MrQuincle »

wlievens wrote: post #26
MrQuincle wrote: post #25

Interesting article. Some things are weird. I don't know why a support vector machine is ranked better than Bayesian nets, or why they are both worse than ensemble methods w.r.t. interpretability. However, I think the human should not be in the loop. The network should have another semantic layer that serves communication. It can be done from the ground up like Steels or Vogt have been doing. In other words, yes we n…

Is that Luc Steels you are referring to? I took a couple classes from him at uni a little over tem years ago. What is he doing lately that you refer to?

Yes, definitely.

I don't think there has been much progress in his work though. :-) I asked him a few months ago if he saw some promising inroads to embodied cognition, but it didn't seem like it. Still it's a part of AI that's neglected on the moment.

PS: Of course there is research, just not mainstream. Der, Parisi, Tani, Bongard, Prokopenko.

Re: Is artificial intelligence permanently inscrutable?

#37

Post by jerf »

cscurmudgeon wrote: post #22
semi-extrinsic wrote: post #16

Earlier quoted context omitted.

If someone wants an analogy: Nobody understands exactly how painkillers (aspirin, paracetamol etc.) work on the molecular level. Yet they are generally held to be quite useful. Edit: electroconvulsive therapy is an even more extreme example; we have no clue how it works, but it's very effective on severe depression. We only know it works because some Italians back in the olden days, before ethics committees were inve…

So how do we get the first sick volunteers for Dr. Neural Network?

AI's been in medical diagnosis for a long time and many people have been treated by it already. There was a news article a month or two ago from IBM for using Watson to do it, and it struck me as obviously a PR piece because "an AI system diagnosed a patient" is breaking news somewhere circa 1980, not now. Its scientific content was zero.

Re: Is artificial intelligence permanently inscrutable?

#38

Post by hyperion2010 »

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of issues, mitigated only by the fact that we have inherited developmental programs that have been subjected to a huge variety of adverse situations that have rigorously tested their performance (by killing anything that failed).

Re: Is artificial intelligence permanently inscrutable?

#39

Post by euske »

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but that doesn't stop them experimenting because they still get something that looks like a result. And hey, they can get paid for it.

Eventually it might grow into a full-fledged science, but it will probably take an awful lot of time.

Re: Is artificial intelligence permanently inscrutable?

#40

Post by DonaldFisk »

jerf wrote: post #37
cscurmudgeon wrote: post #22

Earlier quoted context omitted.

So how do we get the first sick volunteers for Dr. Neural Network?

AI's been in medical diagnosis for a long time and many people have been treated by it already. There was a news article a month or two ago from IBM for using Watson to do it, and it struck me as obviously a PR piece because "an AI system diagnosed a patient" is breaking news somewhere circa 1980, not now. Its scientific content was zero.

Indeed. But circa 1980 they would have been using expert systems, which typically were able to justify their reasoning.
Post reply on HN