Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keras vs PyTorch

deepsense.ai

Post reply on HN
Print view View original

31–40 of 119 posts

Re: Keras vs PyTorch

#31

Post by hak8or »

maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

I totally agree, the lack of documentation via types and lack of smart autocomplete (which I rely on very heavily for API discovery) is not only why I never got into tensor flow, it's also why I never got into languages like python. I even went so far as to use typescript instead of Javascript.

I do believe c# has some machine learning libraries, but afaik they aren't anywhere near the level of tensor flow or keras.

Re: Keras vs PyTorch

#32

Post by microtonal »

formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't.

Use the right tool for the job. Keras can get you to a working model faster. However, I am not sure what the current situation is, but in the past it was not possible to dump and freeze Keras' Tensorflow graphs. This can be a problem if you want to embed a model in a non-Python application.

This attitude of "real deep learning engineers use Tensorflow"

Real engineers use whatever they need to use. But I think that you are overstating the difficulty of Tensorflow. Over the last 6 months, we have hired a couple of students for a research project. Since we standardized on Tensorflow, they had to implement new models in Tensorflow. All of them were up to speed in Tensorflow pretty quickly (they mostly do RNNs and seq2seq learning).

Re: Keras vs PyTorch

#34

Post by amelius »

formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

> Kind of reminds me of assembly programmers that thought C wasn't for l33t 10xx pwner programmers.

The problem with TensorFlow is mainly that you, as a user, have to build a data-dependency graph. This is something a C compiler can do very well, but Python is not so suitable for that.

So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their own language, where the whole data-flow graph determination could be implicit, and not a concern for the programmer.

However, computing a data-flow graph as-you-go (by the library, not the user), like (I think) is done in some libraries, is quite a good approach, since the overhead is quite small (percentage-wise) compared to the large tensor operations that can be performed in highly optimized code.

Re: Keras vs PyTorch

#35

Post by novaRom »

stared wrote: post #5
gjmveloso wrote: post #3

Why not mxnet[1]? Or even better, Gluon[2]? [1]: http://mxnet.incubator.apache.org [2]: https://gluon.mxnet.io

(Another author here). It is explicitly explained in the text. :) tl;dr: not nearly as popular (which means: less tutorials, less documentation, less examples, less integration with other systems, less community support for development or discussions) Sure, all frameworks do have some goal and once one is confident in DL, may be a good choice. As you see from the plots there - MXNet is very fast for some applications…

mxnet is not just fast for "some" applications. It consistently outperforms many frameworks especially in realm of compute intensive convolutions.

Re: Keras vs PyTorch

#36

Post by cttet »

stared wrote: post #26
cttet wrote: post #22

I just like Tensorflow better. For building new models, the graph is complex and errors are unavoidable. There is a separate compile time for Tensorflow and errors will be found before the data come in. Tried pytorch before, the error messages are usually not helpful at all and often leads to clueless debugging for hours. For trying out deep learning, or build on existing models, pytorch or keras may be easier to gra…

It is an interesting perspective, but my experience is exactly the opposite. In Theano debugging was awful. TF felt like a breeze until it didn't. When I jumped on PyTorch - it TF started feeling confusing by comparison. Errors exactly in the defective lines, possibility to print everywhere (or using any other kind of feedback / logging intermediate results). For using models it may note matter that much (though, aga…

I am not sure what is your programming style in Pytorch. As people recommended and in most tutorials I see the sequential approach, where a small mistake of data preprocessing would lead to clueless errors in a completely irrelevant line.

YOLO is a quite standard feed-forward model in my opinion. I mean the math part, which I am more concerned with.

I have never used Theano before, my idea from it is that Tensorflow followed its static graph approach.

Re: Keras vs PyTorch

#37

Post by gaius »

maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

But I'd wish they were written in some language with static typing

CNTK has a C++ API but the documentation is unfortunately just "read the header file" https://docs.microsoft.com/en-us/cognitive-toolkit/cntk-libr...

Also Python obv, or use it as a backend to Keras (in R).

Re: Keras vs PyTorch

#38

Post by thewizardofaus »

Ar-Curunir wrote: post #14
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

> Kind of reminds me of assembly programmers that thought C wasn't for l33t 10xx pwner programmers. It's funny because this the same attitude C/C++ programmers have towards developers using other languages now...

Pffft what are programming languages??? If you aren't writing code in straight up binary then you aren't a real h@k3r

Re: Keras vs PyTorch

#39

Post by mlthoughts2018 »

laingc wrote: post #20
zerostar07 wrote: post #17

While we are at it, which framework is the easiest to tweak at the low-level, e.g. create modified LSTMs etc ?

Pytorch, by a mile. If I had to summarise the frameworks in a few words, they would be: Keras for speed, Tensorflow for production, Pytorch for research.

I would say keras or mxnet for speed and production. PyTorch for research. By this point there are hardly any cases when it’s worth it to descend to lower TensorFlow levels.
Post reply on HN