Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

blogs.microsoft.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#11

Post by varelse »

magicmu wrote: post #7
baq wrote: post #2

quoted performance numbers on multiple GPUs leave other frameworks in the dust. where's the catch?

I was thinking the same thing, it almost looks a little too good to be true -- although it does kinda make sense given the focus on GPU-based clusters. I wonder how this compares to Baidu's warp-ctc [1]. They don't really seem to be the same thing, and maybe I'm missing something since I'm just starting to get into ML, but it seems to be conspicuously absent from this writeup. [1] https://github.com/baidu-research/wa…

1-bit SGD and insanely high minibatch sizes (8192) it would appear: which drastically reduces communication costs, making data-parallel computation scale.

If so, while very cool, that's not a general solution. Scaling batch sizes of 256 or lower would be the breakthrough. I suspect they get away with this because speech recognition has very sparse output targets (words/phonemes).

Too bad the code below isn't open-source because they got g2 instances with ~2.5 Gb/s interconnect to scale:

http://www.nikkostrom.com/publications/interspeech2015/strom...

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#12

Post by hpenedones »

sharms wrote: post #4

This is great news - I was looking at Tensorflow the other day, but on Windows / OSX it doesn't take advantage of my GPUs. For my desktop I was stuck as my new 3440x1440 monitor doesn't, for the time being, work with Xorg's intel driver. Hopefully this is a viable alternative, I would love to see a online course in machine learning leveraging this. I found http://research.microsoft.com/en-us/um/people/dongyu/CNTK-Tu.…

Apparently there is also a CNTK book available in pdf (~150 pages) at: http://research.microsoft.com/pubs/226641/CNTKBook-20160121....

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#14

Post by santoshalper »

tianlins wrote: post #13

They claimed to be "the only public toolkit that can scale beyond single machine". Well, mxnet ( https://github.com/dmlc/mxnet ) can scale across multiple CPUs and GPUs.

Multiple machines implies scaling across a network or backbone - multiple logical computers. As I understand it at least.

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#15

Post by dufalop »

csvan wrote: post #9

First TensorFlow, then Baidu's Warp-CTC, and now Microsofts CNTK. It is a very, very exciting time for open source machine learning indeed.

Definitely !

I'm looking forward to a review of the major systems available by someone who has a clue (i.e. not me). The investment to learn one of these is great enough that it will be worth some time invested to understand what each can do.

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#16

Post by varelse »

csvan wrote: post #9

First TensorFlow, then Baidu's Warp-CTC, and now Microsofts CNTK. It is a very, very exciting time for open source machine learning indeed.

Except that it appears to require MKL or ACML...

Sigh...

./configure

Defaulting to --with-buildtype=release

Cannot find a CPU math library.

Please specify --with-acml or --with-mkl with a path.

Re: Microsoft releases CNTK, its open source deep learning toolkit, on GitHub

#19

Post by pmelendez »

varelse wrote: post #16
csvan wrote: post #9

First TensorFlow, then Baidu's Warp-CTC, and now Microsofts CNTK. It is a very, very exciting time for open source machine learning indeed.

Except that it appears to require MKL or ACML... Sigh... ./configure Defaulting to --with-buildtype=release Cannot find a CPU math library. Please specify --with-acml or --with-mkl with a path.

At least ACML seems to be open source now too.

"ACML End of Life Notice: We have transitioned our math libraries from a proprietary, closed source codebase (ACML) to open source solutions"[1]

[1] http://developer.amd.com/tools-and-sdks/archive/amd-core-mat...

Post reply on HN