Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keras vs PyTorch

deepsense.ai

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Keras vs PyTorch

#61

Post by Raf_ »

mark_l_watson wrote: post #55

Nice article, and I agree with the explanations of what makes Keras and TensorFlow best for specific use cases. Some history: I have used TensorFlow for years, switched to coding against the Keras APIs about 8 months ago. I wish I had more experience with PyTorch, but I just have the time right now to do more than just play with it. One suggestion to the authors: the benchmark figures are interesting, but I wish you…

Glad you like the article and thanks for suggesting researching the CPU usage across these frameworks - it's something worth looking into.

Re: Keras vs PyTorch

#62

Post by inputcoffee »

formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

As others are pointing out, TF isn't that hard.

Or, rather, it is hard but the difficulty is from getting an intuition for what part of this weird multi layer net is producing this weird behavior and is it an artefact or something interesting, and is the connectivity complete and is should I change the learning rate and activation functions?

The real reason to use Tensorflow is the same reason you might use a Go framework instead of Rails: in your heart you have this hope that this thing will one day grow into a really large project and support lots of people and that will be easier with this scalable, optimized code.

Its not even that you'll hit Google scale, its that you'll hit popular scale and still serve the whole thing out of your Digital Ocean droplet.

Re: Keras vs PyTorch

#63

Post by inputcoffee »

Raf_ wrote: post #2

Author here - the article compares Keras and PyTorch as the first Deep Learning framework to learn. It explores the differences between the two in terms of ease of use, flexibility, debugging experience, popularity, and performance, among others. If you have experience with learning, or teaching Deep Learning with PyTorch or Keras, we’d love to hear your thoughts about them.

I appreciate how you focus on just two, which are the state of the art in your opinion.

I find it less useful to see comparisons of "top 50 deep learning frameworks for 2018" which include esoteric stuff that is only there for sake of completeness.

This way a person branching out from Tensorflow (I assume its Tensorflow) knows which two frameworks to try out, and what to look for.

Re: Keras vs PyTorch

#64

Post by Shoue »

maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

I fully agree with you. While I can't speak for Keras, after reading [1] it seems Tensorflow would very much benefit from strong types. You'd easily be able to catch most of the errors presented in the article with higher-kinded and linear types, and maybe you don't even need that much power.

[1]: https://jacobbuckman.com/post/tensorflow-the-confusing-parts...

Re: Keras vs PyTorch

#65

Post by tomrod »

thewizardofaus wrote: post #38
Ar-Curunir wrote: post #14

Earlier quoted context omitted.

> Kind of reminds me of assembly programmers that thought C wasn't for l33t 10xx pwner programmers. It's funny because this the same attitude C/C++ programmers have towards developers using other languages now...

Pffft what are programming languages??? If you aren't writing code in straight up binary then you aren't a real h@k3r

I think there is an emacs command to convert between that and butterfly wing flaps directing cosmic rays to flip bits.

(h/t xckd)

Re: Keras vs PyTorch

#66

Post by rahimnathwani »

inputcoffee wrote: post #62
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

As others are pointing out, TF isn't that hard. Or, rather, it is hard but the difficulty is from getting an intuition for what part of this weird multi layer net is producing this weird behavior and is it an artefact or something interesting, and is the connectivity complete and is should I change the learning rate and activation functions? The real reason to use Tensorflow is the same reason you might use a Go fram…

"Its not even that you'll hit Google scale, its that you'll hit popular scale and still serve the whole thing out of your Digital Ocean droplet."

Are you saying that model inference is slower or less efficient for a model built and trained in Keras, than the same model architecture built directly in tensorflow?

Re: Keras vs PyTorch

#67

Post by stared »

cttet wrote: post #36
stared wrote: post #26

Earlier quoted context omitted.

It is an interesting perspective, but my experience is exactly the opposite. In Theano debugging was awful. TF felt like a breeze until it didn't. When I jumped on PyTorch - it TF started feeling confusing by comparison. Errors exactly in the defective lines, possibility to print everywhere (or using any other kind of feedback / logging intermediate results). For using models it may note matter that much (though, aga…

I am not sure what is your programming style in Pytorch. As people recommended and in most tutorials I see the sequential approach, where a small mistake of data preprocessing would lead to clueless errors in a completely irrelevant line. YOLO is a quite standard feed-forward model in my opinion. I mean the math part, which I am more concerned with. I have never used Theano before, my idea from it is that Tensorflow…

Did you read cost function of YOLO in both languages?

Re: Keras vs PyTorch

#68

Post by wodenokoto »

bitL wrote: post #41
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

Try to run multiple models/ensemble training on many computers with many GPUs to pick up the best performing model or combo. TensorFlow so far has probably the easiest approach for it. That might be reason for the attitude "real deep learning engineers use Tensorflow", as other approaches either don't scale that well or you can't even model something you need for your bleeding-edge billion $-making approach, despite…

Funny, I dumped tensor flow a few years ago because it wasn't possible to do bleeding edge stuff.

Re: Keras vs PyTorch

#69

Post by Raf_ »

phonebucket wrote: post #19

I don't believe I would ever discourage anyone from using any particular framework. The skills learnt from one are highly transferable, so it doesn't matter too much which framework you start with. Also, with eager execution, Tensorflow has become much more accessible to new users. Having said that, the world would likely be a better place if everyone just used PyTorch :)

These are great points, thanks.

Re: Keras vs PyTorch

#70

Post by jacquesm »

bitL wrote: post #13
Raf_ wrote: post #2

Author here - the article compares Keras and PyTorch as the first Deep Learning framework to learn. It explores the differences between the two in terms of ease of use, flexibility, debugging experience, popularity, and performance, among others. If you have experience with learning, or teaching Deep Learning with PyTorch or Keras, we’d love to hear your thoughts about them.

I agree, I also use Keras for stable complex models (up to 1000 layers) in production and PyTorch for fun (DRL). However, if I want to run a distributed training optimization with minimum setup, whether I like it or not, the simplest way is to use TensorFlow's Estimator model and some pre-baked environment like SageMaker. Horovod or CERNDB/Keras require a bit more setup/devops work. The issue with estimators is that…

> I also use Keras for stable complex models (up to 1000 layers)

That sounds interesting, are you at liberty to say what you are doing?

Post reply on HN