Is artificial intelligence permanently inscrutable?
1–10 of 69 posts
Re: Is artificial intelligence permanently inscrutable?
#2I think this issue of "Explainable Machine Learning" and interpretability is just going to get more and more important as ML grows. It will also be important for verifying ML-based systems - another problem area.
See [1] for a discussion of both.
[1] https://blog.foretellix.com/2016/08/31/machine-learning-veri...
Re: Is artificial intelligence permanently inscrutable?
#3Artificial intelligence shows little promise of developing any time soon but still shows promise over long term development.
Re: Is artificial intelligence permanently inscrutable?
#4I can't even understand why deep learning creates better predictions than regular neural nets. How does adding layers change anything?
Re: Is artificial intelligence permanently inscrutable?
#5[deleted]
Re: Is artificial intelligence permanently inscrutable?
#6We're using these things and we're not even sure how they work. Love it.
At least we should have a standard for characterizing their accuracy or something like that ...
Re: Is artificial intelligence permanently inscrutable?
#7We're using these things and we're not even sure how they work. Love it. At least we should have a standard for characterizing their accuracy or something like that ...
> We're using these things and we're not even sure how they work. Love it.
Similar to the human brain in that respect. Ironic that the human brain used itself without understanding how it works to eventually create something it uses without understanding how it works.
Re: Is artificial intelligence permanently inscrutable?
#8I can't even understand why deep learning creates better predictions than regular neural nets. How does adding layers change anything?
I am no expert but I think it allows for a higher order function to be arrived at. An example would be the output of a simple net, where the output is a linear combination of features. This would be extremely shallow and while this will work for some things, there are going to be some instances where this doesn't capture nuanced scenarios.
in a shallow net, maybe college student selection based on sat scores gets a heavy weight/low threshold/whatever. in a shallow linear combo, this will likely always carry a large weight.
in a deeper net, it might be able to learn that SATS are a great predictor except for when X Y Z or some combination of those are some particular value, in which case it might be wholly irrelavent. The deeper it is, the longer it will take to train, but the more it can handle exception cases/trends and approximate reality
Re: Is artificial intelligence permanently inscrutable?
#9I can't even understand why deep learning creates better predictions than regular neural nets. How does adding layers change anything?
In principle a shallow NN (1 hidden layer) can approximate any function. But it has a tendency to overfit and just "memorize" the inputs. The basic idea of adding additional layers, is that the early layers can learn very low-level features of the data, and later layers combine the low-level features into higher-level features. This tends to make the models generalize well.
A standard example is for a face detection algorithm. The first layer will do edge detection, the next layer will combine edges into corners and simple shapes, the next layer will maybe use those shapes to look for features like eyes, noses, mouths, etc., and then the next layer will maybe combine those features to look for a whole face.
I wrote a more detailed answer here:
http://stats.stackexchange.com/questions/222883/why-are-neur...
Re: Is artificial intelligence permanently inscrutable?
#10I can't even understand why deep learning creates better predictions than regular neural nets. How does adding layers change anything?
In general, it allows a better approximation of the solution function for far less hidden neurons. Sure, you could get arbitrarily close using a single hidden layer, but that hidden layer might need to be unfathomably large. Same idea for network topology in multilayer nets - a network could eventually learn to set a lot of the weights to zero, but training is a lot faster and more effective if you know a good problem-specific topology to start with. Deep nets make problems more tractable. Recurrence is the real game-changer, since then you've moved from non-linear function approximators up to Turing completeness (at least over the set of all possible RNNs).