Microsoft Cognitive Toolkit 2.0

Will this be based on Azure notebooks or otherwise cloud based + free?

CNTKs roots are from Speech type data which inherently have a notion of time. The architecture of the toolkit support efficient recurrence from ground up. Also the toolkit focusses on handling large production scale data workloads which implies additional engineering efficiencies built into the toolkit. I am a Microsoft employee.

This sounds awesome, thankyou for doing this! Is there anywhere I can give my email to get updates/a date I can put a reminder in my calendar to check this out?

CNTK uses the LSTM implementation by CuDNN in their official LSTM layer.

TensorFlow has multiple LSTM implementations, such as LSTMCell, BasicLSTMCell, LSTMBlockCell, and also one wrapper for CuDNN, and maybe more. I'm quite confident that in this benchmark, for TensorFlow, they did not use the CuDNN wrapper, which is a bit unfair I would say. Although the CuDNN wrapper in TensorFlow does not support sequences of different lengths but you could overcome this by just ignoring the non-used frames. See here for some more details:

https://stackoverflow.com/questions/41461670/cudnnrnnforward...

Note that you could also provide your own LSTM kernel for TensorFlow, which is what we do in our framework, and then you can get really fast, although our benchmarks are a bit outdated.

https://github.com/rwth-i6/returnn

I think their comparison is a bit unfair.

> Speed. CNTK is in general much faster than TensorFlow, and it can be 5-10x faster on recurrent networks.

This is because they use the CuDNN LSTM kernel but for the TensorFlow comparison they probably did not use the CuDNN LSTM kernel in TensorFlow. See here for some more details: https://news.ycombinator.com/item?id=14473234

> Accuracy. CNTK can be used to train deep learning models with state-of-the-art accuracy.

As well as all other frameworks can do.

> API design. CNTK has a very powerful C++ API, and it also has both low-level and easy to use high-level Python APIs that are designed with a functional programming paradigm.

TensorFlow also has a C++ API.

> Scalability. CNTK can be easily scaled over thousands of GPUs.

Like TensorFlow.

> Inference. CNTK has C#/.NET/Java inference support that makes it easy to integrate CNTK evaluation into user applications.

TensorFlow also has many bindings for other languages.

> Extensibility. CNTK can be easily extended from Python for layers and learners.

TensorFlow can very easily be extended. I did that a lot in our framework (https://github.com/rwth-i6/returnn).

> Built-in readers. CNTK has efficient built in data readers that also support distributed learning.

Just like TensorFlow.

> Identical internal and external toolkit. You would not be compromised in any way because the same toolkit is used by internal product groups at Microsoft.

Ok, maybe here they are better in some sense, although I am not sure that the Google internal version of TensorFlow differs so much. As far as I know, it just has some stuff added for their data centers, for TPU, etc.

Also, the licence of 1-bit SGD is strange in CNTK. Not sure what the state about this is. Last time you could not really use that.

I don't want to downplay CNTK. I really like it. I think it's great that they have really good working CuDNN LSTM wrappers in their default LSTM implementation, and it's better than all the other wrappers (see here: https://stackoverflow.com/questions/41461670/cudnnrnnforward...). The team behind CNTK is really strong. So thank you and congratulation for releasing CNTK 2.0!