Is artificial intelligence permanently inscrutable? • Hacker News Board

Re: Is artificial intelligence permanently inscrutable?

#41

Post by Udo » Fri, Sep 02, 2016, 12:02 AM UTC

euske wrote: post #39

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but that doesn't stop them experimenting because they still get something that looks like a result. And hey, they can get paid for it. Eventually it might grow into a full-fledged science, but it will probably take an awful lot of time.

I disagree about the underlying mechanism being complex. Machine learning algorithms are a class of equation-based systems and strategies for configuring these equations for a specific task. All our math and reasoning tools still apply to this class of algorithms, in principle.

Where we pass into alchemy though is the interplay of these basic components with each other and the parameters they encounter while running, this is where complexity happens. Part of this lies in the very nature of the tasks we use them for: we basically push a cart of raw data in front of a set of "AI" solvers and expect them to do something with it. When that doesn't work, start over, tweak parameters, and try again.

I agree that there is no sufficiently useful intellectual framework for creating these artificially intelligent components, and that shows not only in the uneven success rates and performance, but also in the surprising fact that experts in very different AI systems can usually create components with similar performance characteristics for a given problem, despite using very disparate strategies.

Re: Is artificial intelligence permanently inscrutable?

#42

Post by yummyfajitas » Fri, Sep 02, 2016, 12:34 AM UTC

euske wrote: post #39

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but that doesn't stop them experimenting because they still get something that looks like a result. And hey, they can get paid for it. Eventually it might grow into a full-fledged science, but it will probably take an awful lot of time.

They haven't given up at all. And more importantly, they've developed a huge number of "if it looks good it is good" mathematical theorems.

This means that under many circumstances, you can build a model that satisfies certain abstract properties, test it, and have a high probability of generalizability. I.e. we've circumvented the "understanding" stage.

In fact, I'd say that the core of machine learning (as opposed to merely statistics) is exactly this.

Re: Is artificial intelligence permanently inscrutable?

#43

Post by Houshalter » Fri, Sep 02, 2016, 12:51 AM UTC

This isn't unique to neural networks at all. There was a machine learning system designed to produce interpretable results, called Eureqa. Eureqa is a fantastic piece of software that finds simple mathematical equations that fit your data as good as possible. Emphasis on the "simple", it searches for the smallest equations it can find that works, and gives you a choice of different equations at different levels of complexity.

But still, the results are very difficult to interpret. Yes you can verify that the equation works, that it predicts the data. But why does it work? Well who knows? No one can answer that. Understanding even simple math expressions can be quite difficult.

One biologist put his data into the program, and found, to his surprise, that it found a simple expression that almost perfectly explained one of the variables he was interested in. But he couldn't publish his result, because he couldn't understand it himself. You can't just publish a random equation with no explanation. What use is that?

I think the best method of understanding our models, is not going to come from making simpler models that we can compute by hand. Instead I think we should take advantage of our own neural networks. Try to train humans to predict what inputs, particularly in images, will activate a node in a neural network. We will learn that function ourselves, and then it's purpose will make sense to us.

There is a huge amount of effort put into making more accurate models, but much less into trying to interpret them. I think this is a huge mistake, because understanding a model lets you see it's weaknesses. The things that it can't learn, and the mistakes it makes.

Re: Is artificial intelligence permanently inscrutable?

#44

Post by matk » Fri, Sep 02, 2016, 1:19 AM UTC

Houshalter wrote: post #43

This isn't unique to neural networks at all. There was a machine learning system designed to produce interpretable results, called Eureqa. Eureqa is a fantastic piece of software that finds simple mathematical equations that fit your data as good as possible. Emphasis on the "simple", it searches for the smallest equations it can find that works, and gives you a choice of different equations at different levels of co…

I appreciate the sentiment of your comment, but, what part of a neural net isn't interpretable? Indeed, they do require more careful examination compared to traditional learning techniques. You can examine the receptive field of each node to infer what it detects for.

Re: Is artificial intelligence permanently inscrutable?

#45

Post by chiliap2 » Fri, Sep 02, 2016, 1:44 AM UTC

hyperion2010 wrote: post #38

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of iss…

I strongly agree but I'm left wondering, what would a falsifiable experiment look like in this case?

Re: Is artificial intelligence permanently inscrutable?

#46

Post by dboreham » Fri, Sep 02, 2016, 2:36 AM UTC

sgt101 wrote: post #27

MrQuincle wrote: post #25

Interesting article. Some things are weird. I don't know why a support vector machine is ranked better than Bayesian nets, or why they are both worse than ensemble methods w.r.t. interpretability. However, I think the human should not be in the loop. The network should have another semantic layer that serves communication. It can be done from the ground up like Steels or Vogt have been doing. In other words, yes we n…

Some people, for example medics and civil engineers, are held legally liable for the decisions that they make. If they are to use machines that help them make those decisions (and mostly they would like to due to the terrible business of killing people) then they have to be able to understand what they are being told to do, or they have to trust it enough to bet their futures on it. If the machine is literally infall…

Note the same logic applies to a self-driving car.

Re: Is artificial intelligence permanently inscrutable?

#47

Post by Houshalter » Fri, Sep 02, 2016, 2:46 AM UTC

venning wrote: post #31

The pneumonia-asthma example seems to be an example of a Simpson's paradox [1]. The doctors acted on a strong (accurate) belief about asthma sufferers contracting pneumonia and acted in such a way that the data obscured an actual causal link (asthma as an aggravating factor to pneumonia). This is opposed to the canonical Simpson's paradox where doctors acted on a strong (inaccurate) belief about severe kidney stones…

One method of fixing this would be to have the neural network make 2 predictions. The first would be to predict what decision the doctor would make. The second would be to predict what decision is actually likely to lead to the best outcome.

In cases where it's very likely the doctor would make a different decision, it should flag it for human review.

Re: Is artificial intelligence permanently inscrutable?

#48

Post by Houshalter » Fri, Sep 02, 2016, 2:52 AM UTC

euske wrote: post #39

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but that doesn't stop them experimenting because they still get something that looks like a result. And hey, they can get paid for it. Eventually it might grow into a full-fledged science, but it will probably take an awful lot of time.

I think you mean it's much more like science than math. Not that it's much more like alchemy than science.

Machine learning research is very empirical. With a philosophy of doing lots of experiments and tests, and discovering what methods work best. That's what science is. No scientific field starts off with complete knowledge and understanding, they have to do a lot of experiments to discover the general laws.

Some people dislike empiricism and want pure, provable math. Machine learning isn't a field of mathematics, at least not in practice. But that doesn't mean it's not a science.

Re: Is artificial intelligence permanently inscrutable?

#49

Post by Inlinked » Fri, Sep 02, 2016, 3:28 AM UTC

venning wrote: post #31

The pneumonia-asthma example seems to be an example of a Simpson's paradox [1]. The doctors acted on a strong (accurate) belief about asthma sufferers contracting pneumonia and acted in such a way that the data obscured an actual causal link (asthma as an aggravating factor to pneumonia). This is opposed to the canonical Simpson's paradox where doctors acted on a strong (inaccurate) belief about severe kidney stones…

> Once we have identified such lopsided data and understood its cause, how do you feed that back into your algorithm to correct for it?

This is tackled in the recently popular field of study called 'counterfactual inference'.

http://leon.bottou.org/talks/counterfactuals

Re: Is artificial intelligence permanently inscrutable?

#50

Post by unabst » Fri, Sep 02, 2016, 3:40 AM UTC

chiliap2 wrote: post #45

hyperion2010 wrote: post #38

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of iss…

I strongly agree but I'm left wondering, what would a falsifiable experiment look like in this case?

If it's an assertion, we can test it. "The sky is blue" is an interpretation that can be re-tested. "My car has a flat" can be tested. If it's something that happened, we can't repeat history, so we're left digging up falsifiable evidence. This is the work of detectives when they solve cases, or historians re-evaluating historic facts. You don't re-test blood you no longer have, you're only able to scrutinize the analysis. You don't test again for genocide, you seek documents, records of communication, money trails, etc.

A super casual modern day example would be an assertion like "Joe must be gay". You never know until you ask him, and that is if he's honest with you. But everyday-life is full of gross approximations based on bias and partial information.

But the key, even in Joe's case, is you have to go back to the real source. If all we can do is talk about Joe, we can never know for sure. Much like celebrities we are so certain are gay, but haven't come out openly yet. So as long as computers can't leave their data centers, there really isn't any amount of data that can prove anything to them. They can't experience the truth, because ultimately the truth rests on experience and experience of evidence.