Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keras vs PyTorch

deepsense.ai

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Keras vs PyTorch

#101

Post by mkirklions »

d_burfoot wrote: post #100

Can we please please please not have the kind of framework overproliferation and fragmentation in the Deep ML world that they have in the front-end web world? It's hard enough to learn ML concepts without also having to learn a new ML framework every year.

I was going to disagree, but I am using laravel php (technically backend, but Ive developed a front end for testing)

and.... I'm also using React Native because I dont like Apple and hopefully I can use a friends computer the moment I compile for iphone.

Re: Keras vs PyTorch

#102

Post by eanzenberg »

inputcoffee wrote: post #88
rahimnathwani wrote: post #66

Earlier quoted context omitted.

"Its not even that you'll hit Google scale, its that you'll hit popular scale and still serve the whole thing out of your Digital Ocean droplet." Are you saying that model inference is slower or less efficient for a model built and trained in Keras, than the same model architecture built directly in tensorflow?

Actually, with Tensorflow as a Keras backend, I would expect them to be the same. I am not sure where the performance difference between TF and TF as a backend come from. I do think that pure TF would be easier to scale up over multiple servers etc. but that's only because I don't know how it would work in Keras. Maybe its easy.

Its pretty straightforward to convert a keras model to a tf graph, as long as you used a tf backend in keras.

Re: Keras vs PyTorch

#103

Post by liuliu »

amelius wrote: post #57
sanxiyn wrote: post #47

Earlier quoted context omitted.

> So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their own language, where the whole data-flow graph determination could be implicit, and not a concern for the programmer. You just described Swift for TensorFlow.

Well, apparently :) They say: We believe that machine learning tools are so important that they deserve a first-class language and a compiler. However, I'd like to see some numbers on how more efficient it is to build a graph in advance, given that the lion's share of the computations will be in tensor math anyway (which can be heavily optimized, and is independent of the graph).

The problem is that if you build the graph as-you-go, dataflow graph optimizations cannot be done efficiently (some high level optimizations such as data layout optimization, automatic data / model parallelism etc.). Swift can do all these because the compiler can extract the graph out ahead of time.

Re: Keras vs PyTorch

#104

Post by crucialfelix »

rahimnathwani wrote: post #66
inputcoffee wrote: post #62

Earlier quoted context omitted.

As others are pointing out, TF isn't that hard. Or, rather, it is hard but the difficulty is from getting an intuition for what part of this weird multi layer net is producing this weird behavior and is it an artefact or something interesting, and is the connectivity complete and is should I change the learning rate and activation functions? The real reason to use Tensorflow is the same reason you might use a Go fram…

"Its not even that you'll hit Google scale, its that you'll hit popular scale and still serve the whole thing out of your Digital Ocean droplet." Are you saying that model inference is slower or less efficient for a model built and trained in Keras, than the same model architecture built directly in tensorflow?

I would think the difference would be from the data input pipeline, efficiency in batching, updating online models. The inference itself would be the exact same.

Re: Keras vs PyTorch

#105

Post by robertAngst »

inputcoffee wrote: post #62
formalsystem wrote: post #11

This article echoes my experience as well. I was working on some core NLP models for a larger tech company and wanted to experiment with Keras. I had my models designed within a day and training done within another and had amazing model perf. I was also told that doing it the real way using Tensorflow would be the way to go and I agree with that sentiment if my problem was Google scale which it wasn't. In fact I woul…

As others are pointing out, TF isn't that hard. Or, rather, it is hard but the difficulty is from getting an intuition for what part of this weird multi layer net is producing this weird behavior and is it an artefact or something interesting, and is the connectivity complete and is should I change the learning rate and activation functions? The real reason to use Tensorflow is the same reason you might use a Go fram…

So I'm automating my job on a shitty tiny laptop.

Do you think I'll be able to use DL?

Re: Keras vs PyTorch

#106

Post by supermdguy »

eafkuor wrote: post #45
maaaats wrote: post #27

I don't do much ML (this kind of ML at least), so I know I'm not the target audience for these libraries. But I'd wish they were written in some language with static typing for IDE help. The API interface/tweaks-to-be-done for some of them is enormous, and mostly undiscoverable. I mean, just looking at the "getting started, 30 seconds to Keras"[0], there are so many magic strings and options. Of course, if one is wel…

I've been recently following the course over at fast.ai and I'm having the same issue. The API is, as you mentioned, undiscoverable, and I don't really want to look at the source code, which is kind of unreadable anyway ([0]). The course itself is littered with poorly readable code ([1]) such as: to_np(m.ib(V(topMovieIdx))) Why, just why. Despite all this, I wholeheartedly recommend this course, it demystified DL for…

The terseness is intentional[0], following the idea that "brevity facilitates reasoning". I don't 100% agree with it, but at least there's reasoning behind it instead of just laziness.

[0]: https://github.com/fastai/fastai/blob/master/docs/style.md

Re: Keras vs PyTorch

#107

Post by Smerity »

visarga wrote: post #97
droidist2 wrote: post #95

Earlier quoted context omitted.

I've been using a QRNN in PyTorch, is this similar to a TCN? https://github.com/salesforce/pytorch-qrnn

No, TCN is similar to WaveNet (dilated convolutions + masking the future + residual connections). It's a plain convnet, not an LSTM with a twist. That's why it runs efficiently in parallel on GPUs, like image processing convnets.

Actually, yes, the QRNN has all of those features.

First figure from our paper: how the LSTM with a twist allows for the equivalent speed of a plain convnet by running efficiently in parallel on GPUs, like image processing convents.[1]

Best of all, as it's only an "LSTM with (these) twists", it's drop-in compatible with existing LSTMs but can get you a 2-17 times speed-up over NVIDIA's cuDNN LSTM - essentially speed equivalent to the TCN or WaveNet speed-up.

That's why Baidu implemented QRNN in their production Deep Voice 2 neural text-to-speech (TTS) system[3].

This isn't to say TCN or QRNN is better, simply that it's dangerous to flat out say _no_ if you're not actually certain or don't correctly recall the underlying information.

Disclaimer: I'm the co-author of the QRNN.

Double disclaimer: The TCN paper cites the QRNN but decides not to test against it. They also show results over one of my datasets.

[1]: https://www.semanticscholar.org/paper/Quasi-Recurrent-Neural...

[2]: https://github.com/salesforce/pytorch-qrnn

[3]: https://www.semanticscholar.org/paper/Deep-Voice-2%3A-Multi-...

Re: Keras vs PyTorch

#108

Post by jbgordon »

Love Keras. If you like, you can also use the Keras API inside TensorFlow (as tf.keras). We recently published this guide w/ more info - https://www.tensorflow.org/versions/r1.9/programmers_guide/k... - and are working on a few more examples for the v1.9 release in a couple weeks.

Optionally, you can also use tf.keras in combination w/ eager execution, enabling you to write code like this: https://colab.research.google.com/github/tensorflow/tensorfl...

Re: Keras vs PyTorch

#109

Post by kajecounterhack »

dekhn wrote: post #59

I don't know about how many people external to Google know about tf.estimator, but it's where most people who aren't building complicated custom architectures should be starting. Keras is nice, it's easy to use, but I wouldn't use it to design build and run a massive productive pipeline. tf.estimator is just that.

+1 for the estimators API.

Re: Keras vs PyTorch

#110

Post by rjdagost »

minimaxir wrote: post #83

Speed is one thing, but the key value proposition of Keras for me that rarely comes up in these comparisons are Keras’s native utility functions, including easy and correct text tokenization/padding, easy OHE of categorical variables without using sklearn, and easy model saving/loading from an .hdf5 file. (Although I am not an expert on PyTorch and not as familiar with the ETL pipeline for that)

I agree strongly with the ease of model saving / reloading in Keras. I found this basic functionality to be exasperatingly difficult and cumbersome in TensorFlow.
Post reply on HN