Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Convolutional Neural Networks Work

brohrer.github.io

Post reply on HN
Print view View original

21–30 of 54 posts

Re: How Convolutional Neural Networks Work

#21

Post by danieltillett »

lettergram wrote: post #18
drewm1980 wrote: post #17

I am supportive of clear explanations of some of the building blocks, but I worry repeatedly describing things as being grade school math level gives the wrong impression about the actual learning curve for getting up to speed on working with CNN's. Yes, the building blocks are easy to understand, but actually understanding why a given network structure, or optimization technique isn't working, is a black art. And if…

I setup an automated script to setup a AWS g2 instance, train my neural net using tensorflow, copy my model to my personal computer, and spin down. It costs like $5-$10 to train/test most neural network models. My most expensive model cost like $100 and required a ton of time and resources. It took like 4 days or something. You really dont need $2k workstation.. Of course, for personal use I do have a gtx 1080 becaus…

Would you mind sharing this script? I am thinking of investing in the 2K workstation, but this sounds like a much more efficient way to go.

Re: How Convolutional Neural Networks Work

#22

Post by [deleted] »

DAddYE wrote: post #20
spiderfarmer wrote: post #10

Question: I have a database with 1.000.000 vehicle pictures, organized by make and model. What would be the easiest way to play with this data, so that I can train it to predict the make / model? I don't want to reinvent the wheel now so much tutorials are written and software is being released. What would be the easiest way to start?

OT, I'm interested in this field as well. Are all these pictures USA cars or international? Do you mind disclose where you got them? Thanks!

[deleted]

Re: How Convolutional Neural Networks Work

#25

Post by lettergram »

danieltillett wrote: post #21
lettergram wrote: post #18

Earlier quoted context omitted.

I setup an automated script to setup a AWS g2 instance, train my neural net using tensorflow, copy my model to my personal computer, and spin down. It costs like $5-$10 to train/test most neural network models. My most expensive model cost like $100 and required a ton of time and resources. It took like 4 days or something. You really dont need $2k workstation.. Of course, for personal use I do have a gtx 1080 becaus…

Would you mind sharing this script? I am thinking of investing in the 2K workstation, but this sounds like a much more efficient way to go.

I tried to get it released before, but was shut down by the "open source office". So I can't give you the exact script. However, h2o has a script that launches a cluster, that's very similar:

https://github.com/ledell/h2oEnsemble-benchmarks/blob/master...

You can also do stuff like following in python to execute some code:

    os.system("scp script_demo.py " + sys.argv[1] + ":/home/ubuntu/")
    os.system("ssh " + sys.argv[1] + " sudo chmod 777 /home/ubuntu/script_demo.py")
    os.system("ssh " + sys.argv[1] + " './script_demo.py'")
    os.system("scp " + sys.argv[1] + ":/home/ubuntu/output/* ./output/")
Then it will execute, after it executes - shut it down.

Re: How Convolutional Neural Networks Work

#26

Post by mrfusion »

lettergram wrote: post #18
drewm1980 wrote: post #17

I am supportive of clear explanations of some of the building blocks, but I worry repeatedly describing things as being grade school math level gives the wrong impression about the actual learning curve for getting up to speed on working with CNN's. Yes, the building blocks are easy to understand, but actually understanding why a given network structure, or optimization technique isn't working, is a black art. And if…

I setup an automated script to setup a AWS g2 instance, train my neural net using tensorflow, copy my model to my personal computer, and spin down. It costs like $5-$10 to train/test most neural network models. My most expensive model cost like $100 and required a ton of time and resources. It took like 4 days or something. You really dont need $2k workstation.. Of course, for personal use I do have a gtx 1080 becaus…

What kind of models do y'all make?

Re: How Convolutional Neural Networks Work

#28

Post by viraptor »

lettergram wrote: post #25
danieltillett wrote: post #21

Earlier quoted context omitted.

Would you mind sharing this script? I am thinking of investing in the 2K workstation, but this sounds like a much more efficient way to go.

I tried to get it released before, but was shut down by the "open source office". So I can't give you the exact script. However, h2o has a script that launches a cluster, that's very similar: https://github.com/ledell/h2oEnsemble-benchmarks/blob/master... You can also do stuff like following in python to execute some code: os.system("scp script_demo.py " + sys.argv[1] + ":/home/ubuntu/") os.system("ssh " + sys.argv[1…

Even if it may not matter in this case specifically, this is a terrible example. "sudo chmod 777" in public code is basically "I don't know what I'm doing, but go on, do the same thing yourself" :(

You don't need sudo, because it's your file. You need just "chmod 755", not "777". And you don't need chmod in the first place - just run "python script_demo.py".

Re: How Convolutional Neural Networks Work

#29

Post by polite_cancer »

mrfusion wrote: post #16
polite_cancer wrote: post #14

Earlier quoted context omitted.

Oh, if you don't want to get into the nitty gritty of it, you should use Digits ( https://github.com/NVIDIA/DIGITS ). This is the easiest way to setup a CNN and train it with your sample images (at least compared to Caffe, Tensorflow, and Theano). I say that because it's all GUI based! Real convenient.

Wow that is cool! What if I wanted to make a model to detect cats? Can I just load a bunch of pictures from the web? Do I need negative examples?

Good question. Detecting just one class (in this case, cats) will require negative examples. Finding good negatives is somewhat of challenging task because they should be pretty comprehensive, but if you create an account on image-net (http://image-net.org/), then you can download thousands of images.

Any questions, feel free to pm me.

Re: How Convolutional Neural Networks Work

#30

Post by bilbobeer »

Try them all! And then tell us what you think.

    Caffe
    CNTK
    Deeplearning4j
    TensorFlow
    Theano
    Torch
    Many others
About 5+ years ago I was using ML for music transcription problem's, I largely completed my project. I was satisfied I could take any foreign track and generate a visual music sheet so I could play any song I wish. ( I'm rather tone death and not good enough to play by ear, but I'm very good at playing from sheet music.

Five years ago I used MATLAB all can be gotten from torrent, all of matlab has people who provide all the library's for CNN,RNN, and all the Algo's you could want. Almost all the algo's in MATLAB can be read in one page, which makes it easy to understand the algo, and makes it easy to dial-in the fine tuning required for ML.

In the past month I ventured back into ML, as I have a desire to apply ML to code generation and dynamic analysis.

I have spent over a month on all of the 'free stuff' available... Try them all! And then tell us what you think.

    Caffe
    CNTK
    Deeplearning4j
    TensorFlow
    Theano
    Torch
    Many others
I have found that for almost all of them you have to spend hours on 'Stack' just to get the installation problems dialed-in, I'm using UBUNTU 16.04, so please ... I have found that NONE of the stuff works out of the box, even the KAPARTHY stuff fails without hours of fine-tuning the library's.

When using MATLAB 5+ years ago I found it to be highly productive, in a few days I was able to choose my ALGO because I could take the dozen most popular models apply them to my data, choose which generated the best results, then I dialed in my +98% and I was good to go.

Now after spending a MONTH on all these Lib's and packages, I still don't even have a single model that can easily migrate to my data.

All is about say tensorflow, but let's be honest its a black box, Theano is great, but its clear they dropped the ball years ago, CAFFE is a pain. The best stuff seems to revolve around PYTHON, which is appropriate for fast modelling, as PYTHON is excellent with abstraction just at the right level for math people.

Most of the stuff coming down from Github appears to be from 2015, the only stuff that appears new is TensorFlow, which of course is GOOGLE which means sure its fun to play and see what their doing, at the end of the day you don't want to be in bed with Google (CIA/NSA)

The rule of thumb is to KEEP-IT-SIMPLE. Keep it Fast, Keep it clear.

Most of the models are all based on the MNIST data, which is ok for a very limited problem set.

IMHO the stuff that's being DUMPED today on the market is a GREAT step backword in productivity. There is NOTHING new that wasn't available in 2012. All that we have now is a dozen different packages that are all hyped by BIG players.

Another item don't even bother to get anything working on windows, as nobody is supporting anything for windows.

CUDA has always worked great with MATLAB even 5+ years ago, all you had to do is "USE CUDA", and your python like MATLAB took care of all the business and off-loaded you work to you GPU, which made the computation 10X. Now the NVIDIA lib's are great, but you have to explicitly use CUDA array's in your code. Even KARPATHY models require filters to go back & forth from GPU/CPU data, as they compress the Pickels/CV differently which is a pain. I would prefer to work in GPU 100%.

Things have NOT gotten more simple. Things have gotten burdensome. Then there are all these languages LUA, JIT, Go, everybody has to start at zero,

PYTHON is now 25 years old and the support for MATH is deep, and PYTHON is almost "APL" like in its power over abstract types. There are people abstracting the GPU but its tough to get the stuff working.

The model seems to be to PUSH people off their private hardware and onto a GOOGLE/AMAZON server for $1/hr to do the training, but anybody that actually has done this stuff knows that it can take weeks of 16+ hour days to find the right training ALGO, then another week to dial it in ( find the right loss-function, and Kernel ), and set the hidden-size,drop-out, and learning-rate, thus in actuality you could eat up 100's of dollars all being FED to google/Amazon.

This is just 'rent' folks instead of OWNING your own computer, you would just become a renter, and of course if your poor then you can't play. I don't like this model.

Even the new laptops have MX980 Nvid Gpu's which are plenty enough.

Tensorflow and its brother's are pretty much designed to be services, which means computation in the cloud.

The real developer thing for the little guy is to get you training model done, and then generate a matrix, and then put it into C, and down load to firmware and now you have an expert system on a $1 CPU, then you can do as you wish, the Amazon model is your trained matrix is held in the CLOUD an everybody uses services from say iPhone/android, ok again its the old nickel paradigm of Oracle, where they tax all transaction by service.

Memory is cheap, simplicity is best.

Post reply on HN