To label so-called causative factors or even actual relationships (in a shifting...virtual...hyperspace) among potential relationships is a separate task than to make meaningful predictions or predictable changes. The Universe is inherently a system-less set of potentials. The strongest system is the one that is indeterminate in its methodologies. Systems are survivors of reduction processes.
Is artificial intelligence permanently inscrutable?
11–20 of 69 posts
Re: Is artificial intelligence permanently inscrutable?
#12Artificial intelligence shows little promise of developing any time soon but still shows promise over long term development.
AI has been ten years away for over 10 years so.. it's just around the corner now!
Re: Is artificial intelligence permanently inscrutable?
#13Ah, one of those rare articles that give the impression of being illuminating while leaving people not familiar with the topic far more confused than they were coming in.
Re: Is artificial intelligence permanently inscrutable?
#14I think this issue of "Explainable Machine Learning" and interpretability is just going to get more and more important as ML grows. It will also be important for verifying ML-based systems - another problem area. See [1] for a discussion of both. [1] https://blog.foretellix.com/2016/08/31/machine-learning-veri...
I really disagree and think the whole "There's no way to gauge results!" meme is low-impact FUD. FUD that isn't particularly dangerous so people that don't know any better just believe it (like python can't be performant because of the GIL or macs are better than PCs or some other inanity).
> As exciting as their performance gains have been, though, there’s a troubling fact about modern neural networks: Nobody knows quite how they work. And that means no one can predict when they might fail.
Nonsense! Cross validation. Develop hypotheses, develop subsets of data to prove or disprove given hypotheses, observe how the network reacts. All of these people complaining about not being able to understand what's going on are either reporters, bloggers, or machine learning dabblers looking to say something seemingly unconventional.
From your linked article, which gives more specifics as to the argument:
> Because ML systems are opaque, you cannot really reason about what they do.
Yes, it is possible to reason about a system even if it is "opaque"; the discipline is called reverse engineering. Or the scientific method.
> Also, you can’t do modular (as in module-by-module) verification.
You can do "modular verification" in a variety of ways. Start with analyzing the behavior of each layer and how that changes as you incorporate more layers. It's beyond the scope of this comment to go into it beyond surface level, but there are a lot of papers written about it, google "analyzing neural network hidden activations" or something.
> And you can never be sure what they’ll do about a situation never encountered before.
Humans can never be sure what they'll do about a situation they haven't encountered. Or engineers. They can simulate the events that they can think of, but we can also do that with a neural network.
> Finally, when fixing a bug (e.g. by adding the buggy situation + correct output to the learning set), you can never be sure (without a lot of testing) that the system has fixed “the full bug” and not just some manifestations of it.
Fixing the "full bug" is often not something that can be done in traditional software development "without a lot of testing". Machine learning works the same way.
If you want if/then statements, use a decision tree. If you want strong accuracy on predictions, use a neural network. It helps to know what you're doing when verifying results. You will run into trouble if you don't know what you're doing, as per common sense.
Re: Is artificial intelligence permanently inscrutable?
#15I can't even understand why deep learning creates better predictions than regular neural nets. How does adding layers change anything?
Think of each layer as an opportunity to perform a level of abstraction or categorization. Concepts are built out of smaller concepts which are built out of smaller concepts; lots of layers allows lots of hierarchy in the concepts.
The first layer might recognize and respond to pixels in particular parts of an image, the next layer will group certain of those pixels-responses together into an abstraction you might call a "line", the next layer will respond to certain groupings of lines and add a level of wiggle-room regarding where the lines are in the image, and the final layer will judge whether a combination of groupings constitutes the letter "A". Or at least, if you spent a bit of time poking at a deep network, giving it slightly different inputs, you might eventually conclude that this is what the layers were doing.
Without layers, you're basically just approximating a simple function or mapping with one level of abstraction.
Re: Is artificial intelligence permanently inscrutable?
#16I think this issue of "Explainable Machine Learning" and interpretability is just going to get more and more important as ML grows. It will also be important for verifying ML-based systems - another problem area. See [1] for a discussion of both. [1] https://blog.foretellix.com/2016/08/31/machine-learning-veri...
I really disagree and think the whole "There's no way to gauge results!" meme is low-impact FUD. FUD that isn't particularly dangerous so people that don't know any better just believe it (like python can't be performant because of the GIL or macs are better than PCs or some other inanity). > As exciting as their performance gains have been, though, there’s a troubling fact about modern neural networks: Nobody knows…
If someone wants an analogy:
Nobody understands exactly how painkillers (aspirin, paracetamol etc.) work on the molecular level. Yet they are generally held to be quite useful.
Edit: electroconvulsive therapy is an even more extreme example; we have no clue how it works, but it's very effective on severe depression. We only know it works because some Italians back in the olden days, before ethics committees were invented, decided to electro-shock a bunch of "crazy people" to see what happened. The reason they tried it was that electroshocking of pigs for slaughter had been observed to give a temporary anaesthetic effect.
Re: Is artificial intelligence permanently inscrutable?
#17We're using these things and we're not even sure how they work. Love it. At least we should have a standard for characterizing their accuracy or something like that ...
> We're using these things and we're not even sure how they work. Love it. Similar to the human brain in that respect. Ironic that the human brain used itself without understanding how it works to eventually create something it uses without understanding how it works.
Except human brains are validated by other human brains to be capable of performing a given task. The validations come in many forms with varying degrees of accuracy for the task required. Ex:
-college degree (Doctor)
-employee referral (engineer)
-FizzBuzz (engineer again)
-licensing exam score (driver)
-popular election (supreme commander of the armed forces)
Re: Is artificial intelligence permanently inscrutable?
#18Earlier quoted context omitted.
I really disagree and think the whole "There's no way to gauge results!" meme is low-impact FUD. FUD that isn't particularly dangerous so people that don't know any better just believe it (like python can't be performant because of the GIL or macs are better than PCs or some other inanity). > As exciting as their performance gains have been, though, there’s a troubling fact about modern neural networks: Nobody knows…
If someone wants an analogy: Nobody understands exactly how painkillers (aspirin, paracetamol etc.) work on the molecular level. Yet they are generally held to be quite useful. Edit: electroconvulsive therapy is an even more extreme example; we have no clue how it works, but it's very effective on severe depression. We only know it works because some Italians back in the olden days, before ethics committees were inve…
Or: nobody understands exactly how your doctor's brain reaches its conclusion about what's wrong with you, but you still trust it.
Re: Is artificial intelligence permanently inscrutable?
#19Earlier quoted context omitted.
> We're using these things and we're not even sure how they work. Love it. Similar to the human brain in that respect. Ironic that the human brain used itself without understanding how it works to eventually create something it uses without understanding how it works.
Except human brains are validated by other human brains to be capable of performing a given task. The validations come in many forms with varying degrees of accuracy for the task required. Ex: -college degree (Doctor) -employee referral (engineer) -FizzBuzz (engineer again) -licensing exam score (driver) -popular election (supreme commander of the armed forces)
The same validation scheme(s) can be applied to AI as well. That doesn't mean their inner workings are any more or less penetrable, just that they pass some minimal threshold of competence.
E.g., AlphaGo is clearly good at Go. Yet it makes moves that can appear inscrutable to a human expert.
(NB: I'm not arguing that inscrutability is or is not an issue for AI systems, just suggesting that third-party validations don't meaningfully address the issue one way or the other.)
Re: Is artificial intelligence permanently inscrutable?
#20No, but Nautil.us, with its mandatory tracking cookies, is.