Artificial Neural Networks for Beginners

I think you are talking about Andrew Ng's course.

I completed it and can't recommend it more highly. It is a really excellent, dense course and Ng is a very good teacher.

https://www.coursera.org/learn/machine-learning

It's still widely used in academia (though I'm extremely glad IPython/Jupyter is being praised by Nature[1]), and it has a lot of "toolboxes" with a reasonable integration between them.

Once you step outside it, you need one library for the machine learning part of your project, another one for the computer vision aspects, yet another one for the reading of different data files. When you get to the 20th library you'd need, plus all the classes and wrappers you have to write on your own to achieve the same result, you start looking MATLAB under a softer light. That's even more for those who are no programming-inclined.

[1] http://www.nature.com/news/interactive-notebooks-sharing-the...

I'd agree that Matlab is great and I honestly thought Matlab would be more expensive. I've done Andrew Ng's class, and I learned a great deal from it. Given that I haven't really figured out what I'd like to do with what I learned from the class I'd prefer to stick with the "lite" math package first till I decide to get more serious.

ANNs are great for the right application. But I'm starting to fear "Deep Learning" is the new "Big Data" buzzword.

I believe ANNs are Turing Complete, meaning they should be able to compute anything (EDIT: + "that is computable by any other Turing Machine"). The questions are, can a training regimen be created to create the right ANN to solve "any" problem, and if so, is it an efficient means to solve that problem?

For example, it's fairly trivial to build an ANN to spit out the right results for a given polynomial function, i.e. "f(x, y, z) = ax + by + cz". Knowing the polynomial function ahead of time, you just generate a ton of input/output sets and feed them into the training of the ANN, and then from there on the ANN will spit them back out.

The problem with that is, you didn't learn anything new. You didn't learn how to solve a new problem. It's somewhat useful for teaching people how to program ANNs, but I personally think it's garbage for teaching how to understand ANNs.

ANNs make more sense when we already have the training data, but we don't know the underlying function that maps input to said outputs. In the trivial case of the polynomial function, if someone were to hand us the training set, we could use an ANN to figure out what the polynomial must be.

Except--for this particular example of a polynomial function--this isn't very efficient. For a polynomial of N terms, you only need N+1 sets of IO to trivially use algebra to determine the function. You can use any of the readily available linear algebra libraries to do such a thing. In fact, I wrote a project for a client that does just that: it uses a basic matrix library to crunch a set of GPS data to create a quadratic formula estimation of curves in roads, so that model can them be resampled, continuously, sans noise.

And if that function is not just a simple polynomial--if, say, it includes sines and cosines and square roots, etc.-- then the ANN is going to have to be large enough to include in it ad-hoc, arithmetic estimations of sine and cosine and square roots sufficient to give the right answers. It might even include several different estimating functions just for sine just because our mystery function requires more than one sine operation. It might even have corner cases where it gets the answer wrong, because you didn't have a sufficiently large data set for it to "figure out" things like the fact that sin(x) is approximately x for small values of x. If one knew the right formula (and yes, that's a big if), it'd be significantly more efficient to write a program that computed the values correctly.

All of this is not to poo-poo on ANNs. ANNs are great tools for when we don't know the function and when the function is sufficiently non-trivial to discover. The polynomial example is like trying to kill a fly on the wall with a swarm of nanonmachines designed to evolve and learn how to construct a flyswatter (which is part of the reason I dislike it as a learning tool). But write traditional code to do Optical Character Recognition, I dare you. ANNs are just highly specialized. Think of setting up your ANN like defining the full width and depth of the space of all possible programs that you'd like to search for the program that solves your problem. You then use feedback to "walk" across that space until you find something that looks like your desired program. We're entering an era where we have the memory and distributed processing capabilities to crank out some rather large ANNs. For some problems, we end up training a computer to write programs for us that we could have written on our own. This can impact the number of requests you can handle in a given amount of time.

Of course, that is not necessarily bad, either. "Throwing money at the problem" is not the wrong solution when you have a lot more money than time. Technology is supposed to serve us, not the other way around. Why spend a week discovering a formula to map your data when you can train an ANN in a few hours? And perhaps you don't have very high requirements for request handling. Maybe you only need to process one image a minute on your particular system. Have at it.

But you really, really need to know that is the case before you jump on the ANN bandwagon. You have to know what you want out of the ANN. If you don't have that ability to look at a set of inputs and express a desired set of outputs, then ANN isn't magic pixie dust that will solve that for you. If you have experts in your particular field telling you that your particular problem cannot be easily modeled, then ANNs might be helpful for you. If you are new to your field and you think "let's try an ANN", you're probably going to have a bad time. If you end up with an ANN that is estimating a relatively trivial program, and you're trying to provide a SaaS offering that is meant to scale to thousands or millions of concurrent users, the ANN approach could seriously harm your ability to scale.

Theano is great! The learning curve can be a little difficult, but once it "clicks", it's nice to work with.