Is this used on Azure Machine Learning? Do they use GPU based machines for it on Azure?
I do not think they do yet. There are some plans for GPU based VMs on Azure though
Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
31–40 of 60 posts
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#32Earlier quoted context omitted.
I was thinking the same thing, it almost looks a little too good to be true -- although it does kinda make sense given the focus on GPU-based clusters. I wonder how this compares to Baidu's warp-ctc [1]. They don't really seem to be the same thing, and maybe I'm missing something since I'm just starting to get into ML, but it seems to be conspicuously absent from this writeup. [1] https://github.com/baidu-research/wa…
1-bit SGD and insanely high minibatch sizes (8192) it would appear: which drastically reduces communication costs, making data-parallel computation scale. If so, while very cool, that's not a general solution. Scaling batch sizes of 256 or lower would be the breakthrough. I suspect they get away with this because speech recognition has very sparse output targets (words/phonemes). Too bad the code below isn't open-sou…
Yep. To elaborate: really big batch sizes can speed up training data throughput, but usually mean that less is learned from each example seen, so time-to-convergence might not necessarily improve (might even increase, if you take things too far).
Training data throughput isn't the right metric to compare -- look at time to convergence, or e.g. time to some target accuracy level on held-out data.
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#33I'm a bit surprised they decided on a CAFFE-like declarative language for specifying neural net architectures[1], instead of offering high-level software components that enable easy composition right from within a scripting language, e.g., like Python in TensorFlow's case.[2] Is there anyone from the Microsoft team here that can explain this decision? -- [1] See examples on https://github.com/Microsoft/CNTK/wiki/CNTK…
Let's hope they decide to target F#.
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#34This is great news - I was looking at Tensorflow the other day, but on Windows / OSX it doesn't take advantage of my GPUs. For my desktop I was stuck as my new 3440x1440 monitor doesn't, for the time being, work with Xorg's intel driver. Hopefully this is a viable alternative, I would love to see a online course in machine learning leveraging this. I found http://research.microsoft.com/en-us/um/people/dongyu/CNTK-Tu.…
What gfx card are you using? If nvidia, you should install nvidia's proprietary drivers for linux -- it would be very surprising to me if they didn't support your monitor's resolution.
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#35Earlier quoted context omitted.
Except that it appears to require MKL or ACML... Sigh... ./configure Defaulting to --with-buildtype=release Cannot find a CPU math library. Please specify --with-acml or --with-mkl with a path.
At least ACML seems to be open source now too. "ACML End of Life Notice: We have transitioned our math libraries from a proprietary, closed source codebase (ACML) to open source solutions"[1] [1] http://developer.amd.com/tools-and-sdks/archive/amd-core-mat...
And, if I recall, MKL is included in Microsoft R Open (previously, Revolution R Open). At least, that's what the instructions here[1] lead me to believe.
[1] https://mran.revolutionanalytics.com/documents/rro/installat...
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#36First TensorFlow, then Baidu's Warp-CTC, and now Microsofts CNTK. It is a very, very exciting time for open source machine learning indeed.
Except that it appears to require MKL or ACML... Sigh... ./configure Defaulting to --with-buildtype=release Cannot find a CPU math library. Please specify --with-acml or --with-mkl with a path.
There's a joke that says those that hate Windows use Linux, and those that love unix use BSD.
I think this recent open source push by microsoft is depending on the truth of that statement. I think they're working under the assumption that the recent push of developers using Linux has more to do with those developers wanting to use a superior open source environment than preferring Unix/Linux as an operating system.
All of their open source stuff is really easy to use, IFF you're also using all their other open source software and Visual Studio.
If you're the type of developer who is only using Linux because it has the least path of resistance to using open source libraries and software, they're making good progress towards getting you back into a microsoft ecosystem.
If you're the type of developer who likes the free software philosophy, they're not trying to grab you, because they feel that's not a sizable portion of the people using Linux.
I think they're probably right.
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#37This is great news - I was looking at Tensorflow the other day, but on Windows / OSX it doesn't take advantage of my GPUs. For my desktop I was stuck as my new 3440x1440 monitor doesn't, for the time being, work with Xorg's intel driver. Hopefully this is a viable alternative, I would love to see a online course in machine learning leveraging this. I found http://research.microsoft.com/en-us/um/people/dongyu/CNTK-Tu.…
> For my desktop I was stuck as my new 3440x1440 monitor doesn't, for the time being, work with Xorg's intel driver.
I think this is now fixed upstream
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1494095
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#38Only Microsoft can open source documentation on github in docx https://github.com/Microsoft/CNTK/tree/master/Documentation/...
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#39Only Microsoft can open source documentation on github in docx https://github.com/Microsoft/CNTK/tree/master/Documentation/...
Looks like they're moving it to GitHub wiki: https://github.com/Microsoft/CNTK/wiki/Config-file-overview
Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub
#40First TensorFlow, then Baidu's Warp-CTC, and now Microsofts CNTK. It is a very, very exciting time for open source machine learning indeed.
Don't forget SystemML[1] and Singa[2], as well as venerable systems like Mahout[3] and SAMOA[4].
[1]: http://systemml.apache.org/
[2]: https://singa.incubator.apache.org/