Similar to the R vs python debate?
The State of Machine Learning Frameworks
61–70 of 201 posts
Re: The State of Machine Learning Frameworks
#62We use Pytorch extensively in our startup. We tackle a lot of new research problems as consultants/partners to help develop products or devise new algorithms/models to solve tasks for our customers. We have never regretted our choice to pick Pytorch. I found the article pretty spot on when comparing Tensorflow and Pytorch. The things that have appealed to me about Pytorch are: 1. Extremely easy to debug and work with…
Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to the creation of code that's difficult to reason about. That may be different now with the various flavors of TF (like TF Eager).
I'll add that it was much easier to install PyTorch with GPU support than it was to install TensorFlow with GPU support - at least that's how it was around November of last year. The PyTorch install was painless, whereas we ended up having to build TF from source to work with out setup. Could be different now as I haven't looked at TF since then.
Re: The State of Machine Learning Frameworks
#63Why engineers like Tensorflow: - More code to check-in (Looks more productive) - More infrastructure, e.g. checkpoints, exporters etc. (Looks like they're doing more work) - Fancy visualizations (Allows them to look impressive while presenting loss plots) - Easier to reuse things others have implemented and still get credit for it (TF model zoo, research repo etc.) Why researchers like pytorch: - Way easier to hack t…
> “More code to check-in (Looks more productive)” Not even bad engineers try to pretend like this is true.
Are you saying tensorflow doesn't have more code to check in or that even bad engineers don't pretend that checking in more code looks productive?
Re: The State of Machine Learning Frameworks
#64If I were to start a theoretical computer vision company today, which would I be better off using?
My very biased opinion: you start with PyTorch because it's easy to develop and debug, and there's no point in having the fastest tools for a model that you can't train properly. Once your model is running, and if/when you start hitting performance bottlenecks, then you consider migrating your model to TensorFlow.
But isn't TF eager mode just as easy to develop and debug, and the migration of TF eager mode to TF static mode is then probably simpler?
Re: The State of Machine Learning Frameworks
#65Anyone has any opinions on TF2.0? They've released it recently, and it seems like it should be much closer to PyTorch now, but I don't know enough to evaluate it properly. https://www.youtube.com/watch?v=EqWsPO8DVXk
TF2.0 (and in particular their recommended tf.keras) is simply a clone of the Pytorch API in most respects. There is no reason to use it vs just using Pytorch, especially as Pytorch now support easy model exporting for running in production.
Re: The State of Machine Learning Frameworks
#66Why engineers like Tensorflow: - More code to check-in (Looks more productive) - More infrastructure, e.g. checkpoints, exporters etc. (Looks like they're doing more work) - Fancy visualizations (Allows them to look impressive while presenting loss plots) - Easier to reuse things others have implemented and still get credit for it (TF model zoo, research repo etc.) Why researchers like pytorch: - Way easier to hack t…
Why engineers like Tensorflow:
With those bullet points, looks like you didn't talk to actual engineers, but rather middle-layer management people.
Re: The State of Machine Learning Frameworks
#67Keras still is the very best in terms of expressive models and end to end workflows. It leans heavily on the design idea that you should deliberately design for end to end use cases and all intermediate abstractions should be building blocks that serve precisely that purpose. This is discussed in [0] which IMO is something that deserves to be more widely talked about in software engineering. Lots of other disciplines of software engineering _say_ you should design this way, but in my experience it’s very rare no matter what discipline you’re in. Take TensorFlow itself. It’s a huge mess with no clear abstractions useful for end to end solutions. Just a hodge podge of disparate APIs and way too many underlying engineering concepts were elevated to abstractions for engineer (instead of user) convenience.
Constraining design by end to end use cases is a remarkably robust and useful process.
PyTorch is way better at having clean engineering abstractions than TensorFlow, but still falls short when things like “forward” or maintaining your own training loop and gradient metadata are necessary concepts for a practitioner’s end to end workflow.
[0]: https://blog.keras.io/user-experience-design-for-apis.html
Re: The State of Machine Learning Frameworks
#68You can just shorten it to "Python dominates research", "C++ dominates industry".
Pytorch has a C++ API now.
Re: The State of Machine Learning Frameworks
#69Earlier quoted context omitted.
> “More code to check-in (Looks more productive)” Not even bad engineers try to pretend like this is true.
Are you saying tensorflow doesn't have more code to check in or that even bad engineers don't pretend that checking in more code looks productive?
Not even bad engineers pretend like more code == more productive. Emphatically so.
Re: The State of Machine Learning Frameworks
#70Earlier quoted context omitted.
Jeremy Howard [0] has some takes here [1], mostly negative if I recall correctly. [0] https://www.fast.ai/about/ [1] https://www.youtube.com/watch?v=J6XcP4JOHmk&t=4152s
Wouldn't the CEO of Fast AI, a library using PyTorch, be slightly biased?
They're also championing TensorFlow in Swift, though.