Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keras vs PyTorch

deepsense.ai

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Keras vs PyTorch

#41

Post by bitL »

formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

Try to run multiple models/ensemble training on many computers with many GPUs to pick up the best performing model or combo. TensorFlow so far has probably the easiest approach for it. That might be reason for the attitude "real deep learning engineers use Tensorflow", as other approaches either don't scale that well or you can't even model something you need for your bleeding-edge billion $-making approach, despite other frameworks being much much simpler/more natural and a joy to use.

Re: Keras vs PyTorch

#42

Post by mlthoughts2018 »

Having used both plain TensorFlow and Keras for some very large image processing production services, Keras wins easily, and interoperates with sprinkling in low-level TensorFlow very well.

Even defining a custom deep CNN for multiple image prediction tasks (so, deep and custom architecture), Keras holds up well — and creating your own layers in Keras is very easy.

Re: Keras vs PyTorch

#43

Post by jaegerpicker »

hak8or wrote: post #31
maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

I totally agree, the lack of documentation via types and lack of smart autocomplete (which I rely on very heavily for API discovery) is not only why I never got into tensor flow, it's also why I never got into languages like python. I even went so far as to use typescript instead of Javascript. I do believe c# has some machine learning libraries, but afaik they aren't anywhere near the level of tensor flow or keras.

Scala and Java both have access to really great ML libraries, MLLib in particular is supposed to be really good. I've used it a little but I'm not really a ML expert to judge.

Re: Keras vs PyTorch

#44

Post by bitL »

stared wrote: post #24
hknd wrote: post #15

Earlier quoted context omitted.

From my opinion: Getting started with Tensor Flow, and having a model designed within a day and training within another is also possible. This mostly depends on your model and your data, and (imho) not on the framework of choice. For all, Keras/PyTorch/Tensorflow, you'll need to learn the API - but if you have any ML background, that should be straight forward.

Yes, for all practical problems data is the biggest challenge. Though, debugging matters. In TF it is easy to get errors and spend a lot of time searching for them. In PyTorch it is straightforward. It matters the most when the network, or cost function, is not standard (think: YOLO architecture). E.g. when I wanted to write some differentiable decision tree it took me way longer in TF (I already knew) than with PyTo…

TensorFlow needs some Deep Learning-based assistant to identify common cause of errors you might see on the AST level. Cryptic errors are its weakness and an AI trained to spot correlations between Python AST and error might be very helpful.

Re: Keras vs PyTorch

#45

Post by eafkuor »

maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

I've been recently following the course over at fast.ai and I'm having the same issue. The API is, as you mentioned, undiscoverable, and I don't really want to look at the source code, which is kind of unreadable anyway ([0]). The course itself is littered with poorly readable code ([1]) such as:

to_np(m.ib(V(topMovieIdx)))

Why, just why.

Despite all this, I wholeheartedly recommend this course, it demystified DL for me.

[0]: https://github.com/fastai/fastai/blob/master/fastai/model.py

[1]: https://github.com/fastai/fastai/blob/master/courses/dl1/les...

Re: Keras vs PyTorch

#46

Post by stared »

k__ wrote: post #40

Is TensorFlow.js a viable alternative? I don't know much about Python :/

Learning Python is way faster than learning Deep Learning, so it shouldn't be an issue.

I am planning organizing a TensorFlow.js bootcamp, but here it is more difficult (as data preprocessing, and debugging in general, is way more difficult in JS than in Python).

Re: Keras vs PyTorch

#47

Post by sanxiyn »

amelius wrote: post #34
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

> Kind of reminds me of assembly programmers that thought C wasn't for l33t 10xx pwner programmers. The problem with TensorFlow is mainly that you, as a user, have to build a data-dependency graph. This is something a C compiler can do very well, but Python is not so suitable for that. So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their ow…

> So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their own language, where the whole data-flow graph determination could be implicit, and not a concern for the programmer.

You just described Swift for TensorFlow.

Re: Keras vs PyTorch

#48

Post by stared »

novaRom wrote: post #35
stared wrote: post #5

Earlier quoted context omitted.

(Another author here). It is explicitly explained in the text. :) tl;dr: not nearly as popular (which means: less tutorials, less documentation, less examples, less integration with other systems, less community support for development or discussions) Sure, all frameworks do have some goal and once one is confident in DL, may be a good choice. As you see from the plots there - MXNet is very fast for some applications…

mxnet is not just fast for "some" applications. It consistently outperforms many frameworks especially in realm of compute intensive convolutions.

See charts. Likely, but still 30% speed boost is not a factor for someone learning DL (then debugging, or training wrong models, can easily give overhead of 5-20x).

Re: Keras vs PyTorch

#49

Post by leecarraher »

For nn's in my experience out of memory, and preprocessing tends to cause an equal number issues as the nn optimization. Which tfrecords and streaming seem to solve. Are there similar object loading facilities in pytorch? Though I have not specified models in keras, since it is now part of tf i presume the formats are compatible.

Re: Keras vs PyTorch

#50

Post by candiodari »

microtonal wrote: post #32
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. Use the right tool for the job. Keras can get you to a working model faster. However, I am not sure what the current situation is, but in the past it was not possible to dump and freeze Keras' Tensorflow graphs. This can be a problem if you want to embed a…

> dump and freeze Keras' Tensorflow graphs

You can get a direct reference to the graphs if you want, that will let you do anything tensorflow lets you do. I think this is what you want:

  # This assumes your model is ready to be called with .predict()
  sess = keras.get_session()
  graph = sess.graph
  graph_dev = graph.as_graph_def()

  frozen_graph = tf.graph_util.convert_variables_to_constants(
      sess, graph_def, nodes_to_output)

  encoded_frozen_graph = frozen_graph.SerializeToString()
Post reply on HN