All of my GPU resources are OpenCL based. The only place I could run it is on AWS. Someone really needs to make a CUDA compiler with an OpenCL backend.
GPU-Accelerated Deep Learning Library in Python
11–20 of 51 posts
Re: GPU-Accelerated Deep Learning Library in Python
#12Needs cool demos of practical applications.
If you execute the code shown in the README it should run an experiment against the MNIST dataset[0]. If it performs as well as other deep learning examples[1][2] you should see a test error rate of less than 1% on classification of handwritten digits. This is pretty impressive, especially considering the fact that the collection contains a few digits that I cannot even recognize myself.
[0] http://yann.lecun.com/exdb/mnist/
Re: GPU-Accelerated Deep Learning Library in Python
#13Looks good!
What performance gain could be expected when using GPU vs CPU for Deep Learning? It'd be great to get some figures.
Re: GPU-Accelerated Deep Learning Library in Python
#14All of my GPU resources are OpenCL based. The only place I could run it is on AWS. Someone really needs to make a CUDA compiler with an OpenCL backend.
I have this same issue and am forced to run all my experiments on AWS. Once you get a workflow down it is pretty easy and the AWS resources provide an acceptable level of performance. Though I suppose if you're just toying around / exploring it can be discouraging to go through all of that and incur AWS fees.
Re: GPU-Accelerated Deep Learning Library in Python
#15Hm. And here I was thinking I'd seen most of the accelerated deep learning implementations. Although this one seems a touch incomplete as far as features go. There are some other for those not aware:
[1] [Python] pylearn2 by the Lisa Lab @ uMontreal. Probably one of the beefiest of the bunch. Has a pretty extensive feature set, but IMO isn't the simplest to use.
[2] [Python] MORB by Sander Dielman. Just restricted boltzman machines, but very nice and intuitive to use. Also built on Theano.
[3] [C++] CUV by the AIS lab at Bonn University in Germany. Not strictly deep learning, but it's a fast CUDA-backed library written in templated C++ with bindings to other languages, including python. It's been used to implement deep learning models.
[4] [Python] DeepNet by Nitish Srivastava and co. at U of Toronto. I don't have as much experience with this one but it's built on cudamat and implements most of the main model types. Interestingly, they've taken the approach of using Google protocol buffers as a sustainable means of defining model configurations.
[5] [MATLAB] DeepLearnToolbox by Rasmus Palm. If matlab is your thing, here you are. Implements most models you are likely to want.
Not all of these developed equally as actively developed, but there is some good stuff above and I haven't found many instances where what I wanted wasn't (somewhat) readily available in at least one. I'm sure I'm forgetting one or two.
[1] https://github.com/lisa-lab/pylearn2
[2] https://github.com/benanne/morb
[3] https://github.com/deeplearningais/CUV
[4] https://github.com/nitishsrivastava/deepnet/tree/master/deep...
Re: GPU-Accelerated Deep Learning Library in Python
#16All of my GPU resources are OpenCL based. The only place I could run it is on AWS. Someone really needs to make a CUDA compiler with an OpenCL backend.
I have this same issue and am forced to run all my experiments on AWS. Once you get a workflow down it is pretty easy and the AWS resources provide an acceptable level of performance. Though I suppose if you're just toying around / exploring it can be discouraging to go through all of that and incur AWS fees.
But you must have an Nvidia card around some where to test your codes out first? It looks like the best bang for the buck is a GTX 760.
Re: GPU-Accelerated Deep Learning Library in Python
#17Earlier quoted context omitted.
I have this same issue and am forced to run all my experiments on AWS. Once you get a workflow down it is pretty easy and the AWS resources provide an acceptable level of performance. Though I suppose if you're just toying around / exploring it can be discouraging to go through all of that and incur AWS fees.
But you must have an Nvidia card around some where to test your codes out first? It looks like the best bang for the buck is a GTX 760.
If I had a 760 I'd likely just run all my experiments right on my device. As it stands my only machine is a Early 2009 macbook pro with a Geforce 9600M. The nice part about pylearn2, and theano is that the symbolic expressions compiled by theano can run on your processor too, albeit much slower. You can always test to make sure 1 to 2 epochs work locally before sending it off to AWS for GPU computation.
I'd be very willing to buy a 760, or even a 770 at their current prices. The only thing holding me back is that i'd have to buy an entire computer in which to place the card. Haha :D
If you're interested in just how fast video cards can be for deep learning as compared to CPUs take a peek at the results on [0]. That is for older model GPUs, and they're an order of magnitude faster. Though as I understand it the Geforece 5xx cards are superior for scientific computing as compared to the 6xx and 7xx series which are more gaming oriented. (May still outperform 5xx due to raw speed at the cost of some additional CPU time). Have a look at the appendix on [1] for more info on Fermi vs Kepler GPUs.
[0] http://deeplearning.net/tutorial/lenet.html#running-the-code
Re: GPU-Accelerated Deep Learning Library in Python
#18Hello Hacker News, this is the Hebel developer. Thanks for the attention and the Github stars!
I'll do my best to answer any questions in the thread. Looking forward to you trying out Hebel and your pull requests ;)
Re: GPU-Accelerated Deep Learning Library in Python
#19Earlier quoted context omitted.
But you must have an Nvidia card around some where to test your codes out first? It looks like the best bang for the buck is a GTX 760.
If I had a 760 I'd likely just run all my experiments right on my device. As it stands my only machine is a Early 2009 macbook pro with a Geforce 9600M. The nice part about pylearn2, and theano is that the symbolic expressions compiled by theano can run on your processor too, albeit much slower. You can always test to make sure 1 to 2 epochs work locally before sending it off to AWS for GPU computation. I'd be very w…
The convolutional neural network code that pylearn2 and the Toronto group use is specifically tuned for GTX580 cards - users have reported factors of 2x-10x slowdown using Kepler series cards. In general, most users (of pylearn2 at least) highly recommend a GTX5xx device.
I personally use a GTX570, and it is pretty decent, though not spectacular. Costwise, it is reasonably priced, and "good enough" for most of the networks I have tried (minus ImageNet...)
A key problem is limited GPU memory in the Fermi series, as it is difficult to fit a truly monstrous network on a single card. Krizhevsky's ImageNet work had some very tricky things to spread it across 2 GTX580s, and the training still took a very, very long time.
Re: GPU-Accelerated Deep Learning Library in Python
#20Hm. And here I was thinking I'd seen most of the accelerated deep learning implementations. Although this one seems a touch incomplete as far as features go. There are some other for those not aware: [1] [Python] pylearn2 by the Lisa Lab @ uMontreal. Probably one of the beefiest of the bunch. Has a pretty extensive feature set, but IMO isn't the simplest to use. [2] [Python] MORB by Sander Dielman. Just restricted bo…
There is also DeCAF, which actually includes a way to load a pretrained ImageNet network based on cuda-convnet. I have had pretty recent success using this blob as preprocessing for image classification, ala http://arxiv.org/abs/1310.1531.
Github is here:
https://github.com/UCB-ICSI-Vision-Group/decaf-release/
My current code as an example of combining sklearn/pylearn2 with DeCAF preprocessing (under the decaf folder, sklearn usage is under previous commits):