Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of Machine Learning Frameworks

thegradient.pub

Post reply on HN
Print view View original

81–90 of 201 posts

Re: The State of Machine Learning Frameworks

#81

Post by usmannk »

UncleOxidant wrote: post #62
_coveredInBees wrote: post #19

We use Pytorch extensively in our startup. We tackle a lot of new research problems as consultants/partners to help develop products or devise new algorithms/models to solve tasks for our customers. We have never regretted our choice to pick Pytorch. I found the article pretty spot on when comparing Tensorflow and Pytorch. The things that have appealed to me about Pytorch are: 1. Extremely easy to debug and work with…

Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to…

> That may be different now with the various flavors of TF (like TF Eager)

Unfortunately, if anything I think it's the opposite. The constant creation and deprecation of TF flavors (tf-eager, tf-slim, tf-learn, keras, tf-estimator, tf.contrib [RIP]) has made reading tensorflow code online somewhat disastrous. Everybody, including the TF team, is using a different API and it's difficult to keep all of them straight. It seems that you're doomed to end up using some combination of many of the above in a way that makes sense to you and your team, adding another confusing model to the pile.

Re: The State of Machine Learning Frameworks

#82

Post by _coveredInBees »

arugulum wrote: post #73

Granted that PyTorch and TensorFlow both heavily use the same CUDA/cuDNN components under the hood (with TF also having a billion other non-deep learning-centric components included), I think one of the primary reasons that PyTorch is getting such heavy adoption is that it is a Python library first and foremost. There're maybe all of two "surprises" I've encountered in all my time using it, if even (1. Gradients are…

Yup, you make some great points and I couldn't agree more. Very recently, I was looking into training an object-detector for a custom problem with not many training examples. One of the classes (hardest one to train from few examples) was "person".

I was able to create a custom detection network for a 3-class problem, load up the COCO pretrained weights for the network, strip out all the other weights at the "head" for all the other COCO classes except for the "person" class and then fine-tune the model on my custom 3-class dataset. The resulting model generalized exceptionally well on people as it was still able to retain a lot of its performance from the COCO pre-training. It was so easy to do all of this. Literally, maybe 10 lines of code, and so easy to figure out since I could introspect the state_dict and the weights file directly in my PyCharm interpreter while working out how to do this.

Re: The State of Machine Learning Frameworks

#84

Post by option »

microtonal wrote: post #52

We are considering to move to PyTorch, we really dislike how the Tensorflow 1.x -> 2.0 transition is handled. For years a lot of stuff has been added to tf.contrib , some things were only in tf.contrib and now that it's dropped in TF a lot of project (including ours) have to do quite large rewrites. Since the last few 1.x iterations, Tensorflow has been complaining that the older RNN layers are deprecated and that we…

that’s what we did. We had a popular open source project using TF 1.x

For it’s successor we chose Pytorch instead of TF 2 and have been very happy with this decision

Re: The State of Machine Learning Frameworks

#85

Post by hhtoyou »

amrrs wrote: post #9

I think tensorflow dominates industry purely because of its capability of exporting the model into a coreml Android model or easy of moving it to production in a GCP environment or in whatever form. Pytorch might have to build a good production pipeline around it to catch up in this game. With fastai module that's built on Pytorch learning and developing Deep Learning solutions have become a lot easier. So there's a…

researchers are like Francois Chollet & many more even jeff dean and it is the greatness of the community that it's pretty open to optimization "Why do researchers love PyTorch? Simplicity. It’s similar to numpy, very pythonic, and integrates easily with the rest of the Python ecosystem. For example, you can simply throw in a pdb breakpoint anywhere into your PyTorch model and it’ll work. In TensorFlow, debugging the model requires an active session and ends up being much trickier. Great API. Most researchers prefer PyTorch’s API to TensorFlow’s API. This is partially because PyTorch is better designed and partially because TensorFlow has handicapped itself by switching APIs so many times (e.g. ‘layers’ -> ‘slim’ -> ‘estimators’ -> ‘tf.keras’). Performance. Despite the fact that PyTorch’s dynamic graphs give strictly less opportunity for optimization, there have been many anecdotal reports that PyTorch is as fast if not faster than TensorFlow. It's not clear if this is really true, but at the very least, TensorFlow hasn't gained a decisive advantage in this area."

common people do want to appreciate & adopt the things that seems fit to the knowledge sphere at present from the researchers. tensorflow approaches are better and respected each and everyone of the community as well in exchange enlightened us new ways of understanding of ml solutions. it have turned into a family “If you want to go fast, go alone. If you want to go far, go together.” and given the assets alphabet have a common man can turn into researchers! e.g. >> https://learn.grasshopper.app take this for example "Learn to code anywhere. Grasshopper is available on iOS, Android, and all web browsers. Your progress syncs seamlessly between devices." << this is the status quo ! it's a gift of a lifetime for generations !

Re: The State of Machine Learning Frameworks

#86

Post by nafizh »

ehsankia wrote: post #50
sails wrote: post #14

Earlier quoted context omitted.

Jeremy Howard [0] has some takes here [1], mostly negative if I recall correctly. [0] https://www.fast.ai/about/ [1] https://www.youtube.com/watch?v=J6XcP4JOHmk&t=4152s

Wouldn't the CEO of Fast AI, a library using PyTorch, be slightly biased?

They used tensorflow in their courses previously but later switched to pytorch.

Re: The State of Machine Learning Frameworks

#88

Post by soup10 »

ehsankia wrote: post #45
machinelearning wrote: post #42

Why engineers like Tensorflow: - More code to check-in (Looks more productive) - More infrastructure, e.g. checkpoints, exporters etc. (Looks like they're doing more work) - Fancy visualizations (Allows them to look impressive while presenting loss plots) - Easier to reuse things others have implemented and still get credit for it (TF model zoo, research repo etc.) Why researchers like pytorch: - Way easier to hack t…

Researcher's codes are historically not very clean and re-usable. It may work fine if you want to hack together something to get data for a paper, but if you want to run a real production service, and don't want to drown in tech debt in a year; and that often means more code, as you imply above. I don't think it's unnecessarily verbose, it's just that it's more structured and scalable.

If by some miracle a comp sci researcher does publish something relevant to industry most people would rather wait for some other poor soul to deconstruct their work and make usable code or a library out of it.

Re: The State of Machine Learning Frameworks

#89

Post by arugulum »

microtonal wrote: post #52

We are considering to move to PyTorch, we really dislike how the Tensorflow 1.x -> 2.0 transition is handled. For years a lot of stuff has been added to tf.contrib , some things were only in tf.contrib and now that it's dropped in TF a lot of project (including ours) have to do quite large rewrites. Since the last few 1.x iterations, Tensorflow has been complaining that the older RNN layers are deprecated and that we…

There has basically only ever been one major API shift, which was the shift away from Variables. And, granted, that happened from 0.3->0.4, about a year after the initial release.

Other than that, I've had next to no issues, and the API has only gotten better over time, with more convenient ways to do things.

PyTorch has a much smaller footprint, and is happy to delegate code to separate libraries (e.g. torchvision), so you run into "all-or-nothing" dilemmas less frequently.

Re: The State of Machine Learning Frameworks

#90

Post by hhtoyou »

stinos wrote: post #66
machinelearning wrote: post #42

Why engineers like Tensorflow: - More code to check-in (Looks more productive) - More infrastructure, e.g. checkpoints, exporters etc. (Looks like they're doing more work) - Fancy visualizations (Allows them to look impressive while presenting loss plots) - Easier to reuse things others have implemented and still get credit for it (TF model zoo, research repo etc.) Why researchers like pytorch: - Way easier to hack t…

Why engineers like Tensorflow: With those bullet points, looks like you didn't talk to actual engineers, but rather middle-layer management people.

a micrsoft guy !
Post reply on HN