What great timing for Pytorch Mobile to just be announced
The State of Machine Learning Frameworks
121–130 of 201 posts
Re: The State of Machine Learning Frameworks
#122Earlier quoted context omitted.
Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to…
> That may be different now with the various flavors of TF (like TF Eager) Unfortunately, if anything I think it's the opposite. The constant creation and deprecation of TF flavors (tf-eager, tf-slim, tf-learn, keras, tf-estimator, tf.contrib [RIP]) has made reading tensorflow code online somewhat disastrous. Everybody, including the TF team, is using a different API and it's difficult to keep all of them straight. I…
Agree overall, but tf.eager doesn't have much to do with the rest of the list.
tf.contrib is just a module where user-contributed code was stored, which included both low-level constructs and higher level APIs. tf.estimator is an abstraction that is mostly used for productionizing models. tf.slim/tf.learn were indeed redundant with keras (a library developped externally), but were necessary steps before keras became part of tensorflow.
Re: The State of Machine Learning Frameworks
#123For a long time cafe was the most popular framework for researchers. Most people don’t even know of that now. My point is researchers using a framework (matlab) does not mean it’s used heavily in industries or even all industries.
To be fair I think Matlab is still heavily used in industry, which I view as a direct result of being so dominant among students and researchers. Maybe not for machine learning, but for controls engineering, signal processing, stuff like that it feels unavoidable.
Personally I think all these deep learning frameworks just haven't had as much time to mature, I have a feeling once they do that the one that dominates academia will eventually dominate industry.
Re: The State of Machine Learning Frameworks
#124Earlier quoted context omitted.
I'm not sure that's a great example, given that AWS mostly runs on RHEL-based OSs and Debian is still preferred for Docker. Ubuntu did not "take over the server market".
RHEL is popular for solutions like running a datacenter mostly because it has a nice enterprise support story. It's what the E in that acronym is for, after all. Ubuntu, meanwhile, is quite popular among us mere mortals who have to fix our own boxen. Debian is popular for Docker images exactly because many of the people trying Docker were already familiar with Ubuntu. Those users quickly ended up wanting smaller imag…
But it was so much fun running Slackware servers and compiling our own minimal kernels
make menuconfig
Re: The State of Machine Learning Frameworks
#125We really enjoyed editing this piece. Just wanted to doubly highlight a few of Horace's (chillee on HN) resources linked at the bottom: Code: https://github.com/Chillee/pytorch-vs-tensorflow Ablation of claims: https://thegradient.pub/p/cef6dd26-f952-4265-a2bc-f8bfb9eb1e... JS interactive charts: https://chillee.github.io/pytorch-vs-tensorflow/
Great piece, you might want to update the article with the mention of PyTorch Mobile that released today: https://pytorch.org/mobile/home/
Thanks! We're currently discussing this with Horace.
Re: The State of Machine Learning Frameworks
#126I have worked as data scientist on a lot of finance domain problems - forecasting default, fraud, conversion probability ect.
Lightgbm library has consistently performed well. I've been interested in how many colleagues instantly jump to neural nets when in my experience this often doesn't beat lightgbm on medium sized datasets not related to text/images.
Re: The State of Machine Learning Frameworks
#127Jax, for those that haven't heard of it, is the thing y'all want.
I talk about Jax in the article. It's very cool, especially if you need higher order derivatives. However, it's not meant to be a full neural network library, and unless Google invests significantly into it, it won't take off significantly imo.
Re: The State of Machine Learning Frameworks
#128That first graph is super-confusing. The Y axis says "Percentage of unique mentions" but it only goes up to 0.7%? Was it meant to be "Fraction of unique mentions".
And then the title is "PyTorch vs Tensorflow", but it never says whether the Y axis is unique mentions of PyTorch or Tensorflow? From the context I guess PyTorch, but come on!
The Y axis should be "Fraction mentioning PyTorch", and the title should be "Papers that only mention PyTorch or Tensorflow" (assuming I have understood this correctly).
Shame it was labelled so badly because it's an amazing graph otherwise!
Re: The State of Machine Learning Frameworks
#129I think tensorflow dominates industry purely because of its capability of exporting the model into a coreml Android model or easy of moving it to production in a GCP environment or in whatever form. Pytorch might have to build a good production pipeline around it to catch up in this game. With fastai module that's built on Pytorch learning and developing Deep Learning solutions have become a lot easier. So there's a…
You can do the same with PyTorch, right? Just export the model as .onnx and import it with whatever inference engine you like.
It's difficult because the graph is defined running Python code, which can basically do anything. As I understand it you have to actually run the model and sort of introspect it to work out what the ONNX graph should be.
Whereas the Tensorflow API actually creates a static graph that can be easily converted to ONNX.
Edit: They talk about this problem in the article:
> Although straightforward, tracing has its downsides. For example, it can’t capture control flow that didn’t execute. For example, it can’t capture the false block of a conditional if it executed the true block.
> Script mode takes a function/class, reinterprets the Python code and directly outputs the TorchScript IR. This allows it to support arbitrary code, however it essentially needs to reinterpret Python.
Re: The State of Machine Learning Frameworks
#130What about unique mentions of keras and/or tensorflow? I'm wondering if papers mention keras only and don't mention tensorflow at all.