How Convolutional Neural Networks Work

One thing that might work well with a CNN is geospatial data (easy to consider as an image).

It comes down to the characterizing architecture of convolutional nets, that is weight sharing, and the assumption on data this makes. If by image one means something where you can expect any pattern (at some level in the hierarchy) being equally likely to occur anywhere across an input dimension, then yes this is true. Personally I would say that this is too narrow of a definition of an image (too great of an assumption), and, interestingly enough, perhaps too broad too. I am not a pro.

[Edit] Too broad in the sense that, intuitively, there is perhaps an implied assumption of continuity of the input function defining the image. Note that such assumptions can be made explicit with various so-called statistical priors incorporated in the network.

In 1965 Cooley-Tukey published their famous FFT paper, funny thing is that at Standard Oil they were using FFT in the 1950's, because it could be used for analyzing 'Russian' atomic bomb seismic signatures in real-time it took some 10+ years before it became public ( openly published ), but it was routine in BIG-OIL in 1950's.

Now jump to CNN, we were doing CNN on Cray's in the 1970's on Seismic 3d data ( acoustic sound waves pressure/sheer ) in order to find oil deposits, its the same stuff I see in CNN algo's of today, and its the exact same stuff we were doing in the 1970's in visualizing seismic data (P-S wave ratios were the color that told you what kind of rock/substance you propagated ).

We used to call this 'around the wheel', I suspect that its good to make the kids think this stuff is all something new, most of the foundation of computational ML was designed in 1950's, hell Von Neuman wrote the book on Cybernetic's, automating the human mind with computers, again 1950's.

Another perspective is given the quality ( poor ) that's being leaked, and given corporate history, I suspect that BIG-OIL, and BIG-NSA of today have stuff that is super good and advanced and most of what they leak to GIT-HUB is just garbage.

In summary keep it simply, learn how it really works, and then dial in your own algorithm, and you too will discover the holy-grail, but remember that edoocation is always 15-20 years lagging industry. It's always been this way, and always will. NSA/CIA has always mandated a competitive edge over the foreign gov's, never assume that anything you get for free, or academia is cutting-edge.

( Lastly George Green developed the Math here, back in 1816-ish, thus the Convolutional in CNN is 200 years old to be exact. )

...

Try them all! And then tell us what you think.

    Caffe
    CNTK
    Deeplearning4j
    TensorFlow
    Theano
    Torch
    Many others

About 5+ years ago I was using ML for music transcription problem's, I largely completed my project. I was satisfied I could take any foreign track and generate a visual music sheet so I could play any song I wish. ( I'm rather tone death and not good enough to play by ear, but I'm very good at playing from sheet music.

Five years ago I used MATLAB all can be gotten from torrent, all of matlab has people who provide all the library's for CNN,RNN, and all the Algo's you could want. Almost all the algo's in MATLAB can be read in one page, which makes it easy to understand the algo, and makes it easy to dial-in the fine tuning required for ML.

In the past month I ventured back into ML, as I have a desire to apply ML to code generation and dynamic analysis.

I have spent over a month on all of the 'free stuff' available... Try them all! And then tell us what you think.

    Caffe
    CNTK
    Deeplearning4j
    TensorFlow
    Theano
    Torch
    Many others

I have found that faor almost all of them you have to spend hours on 'Stack' just to get the installation problems dialed-in, I'm using UBUNTU 16.04, so please ... I have found that NONE of the stuff works out of the box, even the KAPARTHY stuff fails without hours of fine-tuning the library's.

When using MATLAB 5+ years ago I found it to be highly productive, in a few days I was able to choose my ALGO because I could take the dozen most popular models apply them to my data, choose which generated the best results, then I dialed in my +98% and I was good to go.

Now after spending a MONTH on all these Lib's and packages, I still don't even have a single model that can easily migrate to my data.

All is about say tensorflow, but let's be honest its a black box, Theano is great, but its clear they dropped the ball years ago, CAFFE is a pain. The best stuff seems to revolve around PYTHON, which is appropriate for fast modelling, as PYTHON is excellent with abstraction just at the right level for math people.

Most of the stuff coming down from Github appears to be from 2015, the only stuff that appears new is TensorFlow, which of course is GOOGLE which means sure its fun to play and see what their doing, at the end of the day you don't want to be in bed with Google (CIA/NSA)

The rule of thumb is to KEEP-IT-SIMPLE. Keep it Fast, Keep it clear.

Most of the models are all based on the MNIST data, which is ok for a very limited problem set.

IMHO the stuff that's being DUMPED today on the market is a GREAT step backword in productivity. There is NOTHING new that wasn't available in 2012. All that we have now is a dozen different packages that are all hyped by BIG players.

Another item don't even bother to get anything working on windows, as nobody is supporting anything for windows.

CUDA has always worked great with MATLAB even 5+ years ago, all you had to do is "USE CUDA", and your python like MATLAB took care of all the business and off-loaded you work to you GPU, which made the computation 10X. Now the NVIDIA lib's are great, but you have to explicitly use CUDA array's in your code. Even KARPATHY models require filters to go back & forth from GPU/CPU data, as they compress the Pickels/CV differently which is a pain. I would prefer to work in GPU 100%.

Things have NOT gotten more simple. Things have gotten burdensome. Then there are all these languages LUA, JIT, Go, everybody has to start at zero,

PYTHON is now 25 years old and the support for MATH is deep, and PYTHON is almost "APL" like in its power over abstract types. There are people abstracting the GPU but its tough to get the stuff working.

The model seems to be to PUSH people off their private hardware and onto a GOOGLE/AMAZON server for $1/hr to do the training, but anybody that actually has done this stuff knows that it can take weeks of 16+ hour days to find the right training ALGO, then another week to dial it in ( find the right loss-function, and Kernel ), and set the hidden-size,drop-out, and learning-rate, thus in actuality you could eat up 100's of dollars all being FED to google/Amazon.

This is just 'rent' folks instead of OWNING your own computer, you would just become a renter, and of course if your poor then you can't play. I don't like this model.

Even the new laptops have MX980 Nvid Gpu's which are plenty enough.

Tensorflow and its brother's are pretty much designed to be services, which means computation in the cloud.

The real developer thing for the little guy is to get you training model done, and then generate a matrix, and then put it into C, and down load to firmware and now you have an expert system on a $1 CPU, then you can do as you wish, the Amazon model is your trained matrix is held in the CLOUD an everybody uses services from say iPhone/android, ok again its the old nickel paradigm of Oracle, where they tax all transaction by service.

Memory is cheap, simplicity is best.

It's not cars, but mostly tractors. I have been running an online community for almost 10 years now and the images have been submitted and organized by the members.

If you google "Hinton machine learning" on youtube, you will find hinton's lecture's they are non-mathematical, he is a psychologist/math guy, and he is the inventor of almost all this stuff, backprop, drop-out,

You will find his lectures to be very entertaining and easy to understand, being a psychologist whose desire is to make a computer operate like a human brain, he's more interested in how the brain actually works, than hacking ML code.

Hinton describes backprop, why he invented it, and exactly how it emulates the way the human brain works.

Hinton now works at microsoft, he is considered the modern day 'godfather' of DeepLearning/ML