Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keras vs PyTorch

deepsense.ai

Post reply on HN
Print view View original

71–80 of 119 posts

Re: Keras vs PyTorch

#71

Post by bitL »

wodenokoto wrote: post #68
bitL wrote: post #41

Earlier quoted context omitted.

Try to run multiple models/ensemble training on many computers with many GPUs to pick up the best performing model or combo. TensorFlow so far has probably the easiest approach for it. That might be reason for the attitude "real deep learning engineers use Tensorflow", as other approaches either don't scale that well or you can't even model something you need for your bleeding-edge billion $-making approach, despite…

Funny, I dumped tensor flow a few years ago because it wasn't possible to do bleeding edge stuff.

Yeah, for many bleeding edge things you still need PyTorch ;-)

Re: Keras vs PyTorch

#72

Post by bitL »

jacquesm wrote: post #70
bitL wrote: post #13

Earlier quoted context omitted.

I agree, I also use Keras for stable complex models (up to 1000 layers) in production and PyTorch for fun (DRL). However, if I want to run a distributed training optimization with minimum setup, whether I like it or not, the simplest way is to use TensorFlow's Estimator model and some pre-baked environment like SageMaker. Horovod or CERNDB/Keras require a bit more setup/devops work. The issue with estimators is that…

> I also use Keras for stable complex models (up to 1000 layers) That sounds interesting, are you at liberty to say what you are doing?

Complex computer vision classification tasks based on DenseNet/ResNet approaches; those often could be reduced in depth by some Wide ResNet technique. Keras is super easy there and you get a world-class performance after 1 hour of coding and a week of training, when you know what are you doing.

Re: Keras vs PyTorch

#73

Post by mlthoughts2018 »

bitL wrote: post #41
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

Try to run multiple models/ensemble training on many computers with many GPUs to pick up the best performing model or combo. TensorFlow so far has probably the easiest approach for it. That might be reason for the attitude "real deep learning engineers use Tensorflow", as other approaches either don't scale that well or you can't even model something you need for your bleeding-edge billion $-making approach, despite…

Most of the tools that TensorFlow offers for multi-gpu and distributed model training will "just work" directly with Keras models too, or with really minor tweaks. You can even easily mix and match pure TensorFlow code (like explicitly setting the device with a device placement context manager) with Keras code.

See e.g. [0] and [1] linked below.

For model ensembling, it's even easier. After training, in Keras you could simply load your multiple models and create a new Model() object that does nothing but use a merge layer (with mode set to averaging) to average across multiple input models, even if the models share layers or have other crazy constraints. Writing that final ensemble is extremely easy in Keras.

In my experience researching and productionizing very deep Keras models for an image processing use case that has moderately tight performance constraints, Keras has proved to scale extremely well and the code remains dead simple the whole time.

[0]: https://blog.keras.io/keras-as-a-simplified-interface-to-ten... >

[1]: https://www.tensorflow.org/programmers_guide/estimators#crea... >

Re: Keras vs PyTorch

#74

Post by microtonal »

candiodari wrote: post #50
microtonal wrote: post #32

Earlier quoted context omitted.

I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. Use the right tool for the job. Keras can get you to a working model faster. However, I am not sure what the current situation is, but in the past it was not possible to dump and freeze Keras' Tensorflow graphs. This can be a problem if you want to embed a…

> dump and freeze Keras' Tensorflow graphs You can get a direct reference to the graphs if you want, that will let you do anything tensorflow lets you do. I think this is what you want: # This assumes your model is ready to be called with .predict() sess = keras.get_session() graph = sess.graph graph_dev = graph.as_graph_def() frozen_graph = tf.graph_util.convert_variables_to_constants( sess, graph_def, nodes_to_outp…

That didn't work before, but admittedly, the last time I tried was probably 1.5 years ago.

Re: Keras vs PyTorch

#75

Post by _v7gu »

Having used Torch (the Lua library) before, the comparison between the Sequential models seems very absurd. Even the pyTorch documentation gives an almost equivalent model defintion method:

    # Example of using Sequential
    model = nn.Sequential(
      nn.Conv2d(1,20,5),
      nn.ReLU(),
      nn.Conv2d(20,64,5),
      nn.ReLU()
    )

Re: Keras vs PyTorch

#76

Post by mlthoughts2018 »

bitL wrote: post #72
jacquesm wrote: post #70

Earlier quoted context omitted.

> I also use Keras for stable complex models (up to 1000 layers) That sounds interesting, are you at liberty to say what you are doing?

Complex computer vision classification tasks based on DenseNet/ResNet approaches; those often could be reduced in depth by some Wide ResNet technique. Keras is super easy there and you get a world-class performance after 1 hour of coding and a week of training, when you know what are you doing.

I mentioned in another comment [0], but also useful here: most of TensorFlow's tools for distributed model training or multi-gpu training will work out of the box directly on Keras, and distributed training is not at all a reason to directly use TensorFlow over Keras. At worst, you have to add in a tiny bit on TensorFlow code on top of the majority being in Keras, but you would still never need to write a significant amount directly in TensorFlow.

I also work on production systems built around deep ResNet architecture for computer vision tasks, and my team does this using solely Keras, including when we do distributed training.

Just adding this thought in case anyone mistakenly thinks you have to start out all-in using only TensorFlow because you might expect to need distributed training at some point.

[0]: https://news.ycombinator.com/item?id=17416904 >

Re: Keras vs PyTorch

#77

Post by Al-Khwarizmi »

probably_wrong wrote: post #56
Raf_ wrote: post #2

Author here - the article compares Keras and PyTorch as the first Deep Learning framework to learn. It explores the differences between the two in terms of ease of use, flexibility, debugging experience, popularity, and performance, among others. If you have experience with learning, or teaching Deep Learning with PyTorch or Keras, we’d love to hear your thoughts about them.

For what it's worth, here's my experience: My adviser decided (wisely) that we all needed to learn NN, and we settled on Tensorflow. That went... poorly. I've told this before: the Seq2Seq tutorial was designed for an older version of TF, and it triggered a bug that was not fixed because that way to do Seq2Seq was deprecated and a new tutorial was coming "soon". The "tutorial" was also just a code dump with barely an…

Similar story here. I got bitten by that very seq2seq "tutorial", lost a lot of time with it, and haven't used TensorFlow ever since except for reproducing other people's experiments. It's Keras, Torch, DyNet or PyTorch for me.

Re: Keras vs PyTorch

#78

Post by bitL »

mlthoughts2018 wrote: post #73
bitL wrote: post #41

Earlier quoted context omitted.

Try to run multiple models/ensemble training on many computers with many GPUs to pick up the best performing model or combo. TensorFlow so far has probably the easiest approach for it. That might be reason for the attitude "real deep learning engineers use Tensorflow", as other approaches either don't scale that well or you can't even model something you need for your bleeding-edge billion $-making approach, despite…

Most of the tools that TensorFlow offers for multi-gpu and distributed model training will "just work" directly with Keras models too, or with really minor tweaks. You can even easily mix and match pure TensorFlow code (like explicitly setting the device with a device placement context manager) with Keras code. See e.g. [0] and [1] linked below. For model ensembling, it's even easier. After training, in Keras you cou…

Thanks for the links! Do you know how to convert a generator from Keras to an input in estimator, add class weights, custom loss functions, plug-in various Keras-based callbacks as well? I couldn't find any guide for that part.

What do you use to orchestrate distributed training in Keras?

Re: Keras vs PyTorch

#79

Post by nbeleski »

Interesting article. I've been doing some production work with ML and I was wondering which tool would work better in my specific enviroment.

Currently I've been training a CNN model in Keras with good success, and using custom scripts to port it to a TensorFlow model. The .h5 file from Keras helps a lot with this step.

Next step is compiling a shared Tensorflow library so I can deploy the trained model in C++ (project requirement) and this has been a pain in the ass, regardless of framework...

Re: Keras vs PyTorch

#80

Post by sandGorgon »

formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

Did you look at Tensorflow Estimators ? They are a new high-level API with built in support for distributed training.

https://www.tensorflow.org/programmers_guide/estimators

Post reply on HN