41–50 of 88 posts
Does anyone use AMD for deep learning in scientific / industry ? All the libraries for deep learning I have seen require CUDA and NVIDIA is winning by merely being the most popular API. Searching github it looks like they are university assignment projects see https://github.com/search?utf8=%E2%9C%93&q=opencl+deep+learn...
Earlier quoted context omitted.
Not only not trivial to port those but they are constantly will have to play catch up and be at the mercy of NVIDIA in regards to spec. AMD is in a catch 22, support CUDA and be effectively in a constant catch up position, not support it and continue to be ignored by the market at large simply because the momentum NVIDIA has managed to achieve with CUDA over the years.
NVIDIA is playing catch-up with themselves and the shifting market too! Just look at the Maxwell-based Tesla cards that came out of the blue (as if they were an afterthought); I bet Facebook, Baidu, Google, etc. told NVIDIA that Kepler was shit for their use-cases and they did not want to wait for Pascal. Or look at the bizarre Pascal product line where the GP100 does support half precision, but the others don't, not…
As for the half precision, it's pretty much the same thing NVIDIA been doing since Kepler dumping FP64 and FP16, especially FP16 due to the silicon costs.
NVIDIA came out with the Titan and Titan Black with baller FP16 performance and no one seem to care, the Titan X then dropped it and people bought it like it was cupcakes, I'm pretty sure they have pretty good market research that states most people can live without it and those who can't can pay through the nose.
As for Google and Baidu while they are huge I'm not sure how "important" they are, Google can pretty much design their own hardware at this point, and as you mentioned they don't really use the software ecosystem that much as they can write everything from scratch even the driver if need be.
What CUDA gives is a huge ecosystem for 3rd parties and more importantly a lock on developers since that's "all they know", it's not that different than how MSFT got a lock on the IT industry through sysadmins that only knew Windows and even now with Dev Ops and everyone and their mother running Linux they are still an important player.
If the majority of the commercial software is CUDA based, if most researchers and developers are exposed more to CUDA and are more experienced with it NVIDIA has a lock on the market.
I'm not entirely sure how much big of a client or how good of a client Google will be they can demand pretty steep discounts and they are a heartbeat away from building their own hardware probably anyhow.
NVIDIA doesn't want to be locked to 2-3 huge contracts that pretty much dictate how their hardware and software should look like, that's what put AMD in a bind with the console contracts dictating how their GPU's are going to look for a few generations now.
Earlier quoted context omitted.
Those MIOpen benchmarks are a bit dubious, since MIOpen is AMDs own deep learning framework. It's unlikely that code written by AMD is optimal for the Nvidia hardware. To be realistic you need to compare AMD hardware running MIOpen to NV hardware running a framework backed by cuDNN.
It's clearly indicated on the slide that those are Deepbench [1] GEMM and GEMM-convolution numbers. Data for M40, TITAN Maxwell/Pascal and Intel KNL is actually provided by Baidu in their Github repo. [1] https://github.com/baidu-research/DeepBench
What's a good GPU / setup for someone doing deep learning at home? Does anyone have recommendations?
I've been curious about this too. AWS does have K80 instances available for $0.90/hour which isn't too bad for playing around and as long as they keep updating their infrastructure, you can play with the newest stuff versus having to upgrade your own all the time.
I'm just starting off, so my primary concern is speed of iteration and learning. I want to train on and generate audio phonemes, so this will undoubtably take a lot of practice.
What's a good GPU / setup for someone doing deep learning at home? Does anyone have recommendations?
What's a good GPU / setup for someone doing deep learning at home? Does anyone have recommendations?
I'm in Latin America, and wonder which kind of niche could be satisfied with this, for more "normal" kind of customers.
Earlier quoted context omitted.
I've been curious about this too. AWS does have K80 instances available for $0.90/hour which isn't too bad for playing around and as long as they keep updating their infrastructure, you can play with the newest stuff versus having to upgrade your own all the time.
That doesn't sound that bad. I might investigate AWS until putting an investment into my own dedicated hardware. I'm just starting off, so my primary concern is speed of iteration and learning. I want to train on and generate audio phonemes, so this will undoubtably take a lot of practice.
Earlier quoted context omitted.
Some people do, but they're by far the minority. CUDA / cuDNN are well tuned and integrated with the most popular libraries. At this point, in order to get people to switch not only would your hardware have to be faster, but your replacement for cuDNN would have to be better as well. Which is by no means impossible, just difficult.
I disagree; most people use caffe / keras / theano / tensorflow / etc hiding the cuDNN details so end users won't care much. Offering more performance / shorter training loops is a big deal. Your typical ML learning loop with an expensive human regularly waiting for experiment results shows clear benefits.