91–100 of 201 posts
Whatever happened to Julia? Wasn't it supposed to incorporate all these incredible abstractions at the language level and run quickly on GPUs and everything in-between? Is it just lack of adoption or is has it something else?
We use Pytorch extensively in our startup. We tackle a lot of new research problems as consultants/partners to help develop products or devise new algorithms/models to solve tasks for our customers. We have never regretted our choice to pick Pytorch. I found the article pretty spot on when comparing Tensorflow and Pytorch. The things that have appealed to me about Pytorch are: 1. Extremely easy to debug and work with…
Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to…
There will be bad model code in both PyTorch and TensorFlow. The difference is that, bad PyTorch code reads like bad Python code, and I've accumulated a lot of experience reasoning through bad Python code. Bad TensorFlow code can come from any one of of the history of paradigms that TF has gone through, and I don't even know if it's bad or just some funky new TF functionality I'm unfamiliar with.
Earlier quoted context omitted.
Are you saying tensorflow doesn't have more code to check in or that even bad engineers don't pretend that checking in more code looks productive?
Not even bad engineers pretend like more code == more productive. Emphatically so.
However its been my experience that the average engineer believes this. They often aim to push some amount of code (meaningful or not) every few days or so.
The best engineers I know write a lot of code overall but are more interested in ensuring they build the right thing and are ok with not pushing code for a while if they need more time.
Back when we were using TensorFlow, whenever we wanted to try something new, sooner or later we would find ourselves wrestling with its computational graph abstraction, which is non-intuitive, especially for models with more complex control flow.
That said, we are keeping an eye on Swift + MLIR + TensorFlow. We think it could unseat PyTorch for R&D and eventually, production, due to (a) the promise of automatic creation of high-performance GPU/TPU kernels without hassle, (b) Swift's easy learning curve, and (c) Swift's fast performance and type safety. Jeremy Howard has a good post about this: https://www.fast.ai/2019/03/06/fastai-swift/
We really enjoyed editing this piece. Just wanted to doubly highlight a few of Horace's (chillee on HN) resources linked at the bottom: Code: https://github.com/Chillee/pytorch-vs-tensorflow Ablation of claims: https://thegradient.pub/p/cef6dd26-f952-4265-a2bc-f8bfb9eb1e... JS interactive charts: https://chillee.github.io/pytorch-vs-tensorflow/
Why engineers like Tensorflow: - More code to check-in (Looks more productive) - More infrastructure, e.g. checkpoints, exporters etc. (Looks like they're doing more work) - Fancy visualizations (Allows them to look impressive while presenting loss plots) - Easier to reuse things others have implemented and still get credit for it (TF model zoo, research repo etc.) Why researchers like pytorch: - Way easier to hack t…
Why engineers like Tensorflow: With those bullet points, looks like you didn't talk to actual engineers, but rather middle-layer management people.
Earlier quoted context omitted.
My very biased opinion: you start with PyTorch because it's easy to develop and debug, and there's no point in having the fastest tools for a model that you can't train properly. Once your model is running, and if/when you start hitting performance bottlenecks, then you consider migrating your model to TensorFlow.
But isn't TF eager mode just as easy to develop and debug, and the migration of TF eager mode to TF static mode is then probably simpler?
Whatever happened to Julia? Wasn't it supposed to incorporate all these incredible abstractions at the language level and run quickly on GPUs and everything in-between? Is it just lack of adoption or is has it something else?
Though you can already use very clean Pytorch style libraries like Flux and Knet or the Tensorflow bindings to leverage the benefits of Julia for high performance numerical processing on the adjacent tasks such as data preprocessing.
Earlier quoted context omitted.
And simply TensorFlow was there earlier, so people implemented stuff in it. I think there's more inertia in industry, whereas researchers may more easily switch frameworks between two papers.
And Keras was there too which made a lot of people us Keras with TF. I doubt if TF would have risen to this popularity hadn't it been to Keras.