51–60 of 67 posts
Is anybody reading this book? How good is it for a beginner?
The logic within a computer processor is made entirely with nand-gates, and a processor is able to compute any function. Nand-gates have 'functional completeness' and can implement any other gate (AND, OR, NOT, NOR), and any other high level construct with those functions.
Similarly, neurons in a neural net can output a 'NAND' function if set up correctly.
This provides a logical inference of computational completeness of neural-nets
They can compute any function, but the same can be said about splines, polynomials and many other ways to approximate functions. The problem is with an algorithms which will not lead to overfitting and can really reasonably approximate any function.
But yeah, the overfitting problem / approximation is the real issue. I do like seeing the hybrid approaches (Genetic Algorithsm searching random Neural Nets with a little bit of backpropigation for some measure)
Earlier quoted context omitted.
If you lived in the 1800's, would you have described yourself as an automaton made of gears?
Perhaps - it's not a bad analogy. But calling myself a neural network is not an analogy - it's literal.
Earlier quoted context omitted.
As the article says: " If you're a mathematician the argument is not difficult to follow, but it's not so easy for most people. That's a pity, since the underlying reasons for universality are simple and beautiful." Indeed, as a mathematician, the universality of neural networks is obvious to me from their definition. However, this article is explicitly not aimed at mathematicians, and (as far as I can tell) does a g…
As a non mathematician, this is non obvious to me. Thinking about it a bit (I haven't finished reading the article yet)... Since the size of the hidden layer isn't specified, I suppose you could have a hidden layer node for every possible input... So, of course any function is computable with a neural network. Really the magical thing here is finding the smallest set of nodes that computes the function...
I currently have 2 hidden layers, of 12 and 8 and it's the best I've gotten so far.
Is anybody reading this book? How good is it for a beginner?
We used it to prepare a 10-day course on neuronal networks for high school students. For anyone interested (and capable of reading German), see https://github.com/iblech/mathematik-der-vorhersagen for some notes, Python code, and videos.
> Consider the problem of naming a piece of music based on a short sample of the piece. That can be thought of as computing a function. Or consider the problem of translating a Chinese text into English. Again, that can be thought of as computing a function. Or consider the problem of taking an mp4 movie file and generating a description of the plot of the movie, and a discussion of the quality of the acting. Again,…
For example, suppose that the function to be approximated is simply f(x) = x. For any real numbers a Therefore, the translation example works if we assume that the number of different Chinese texts is finite, but otherwise nothing is guaranteed.
Earlier quoted context omitted.
As the article says: " If you're a mathematician the argument is not difficult to follow, but it's not so easy for most people. That's a pity, since the underlying reasons for universality are simple and beautiful." Indeed, as a mathematician, the universality of neural networks is obvious to me from their definition. However, this article is explicitly not aimed at mathematicians, and (as far as I can tell) does a g…
As a non mathematician, this is non obvious to me. Thinking about it a bit (I haven't finished reading the article yet)... Since the size of the hidden layer isn't specified, I suppose you could have a hidden layer node for every possible input... So, of course any function is computable with a neural network. Really the magical thing here is finding the smallest set of nodes that computes the function...