111–120 of 201 posts
PyTorch is easy to use and modify, but Chainer, and by extension cupy (a separate awesome project!) are really, really easy to work with.
Jax, for those that haven't heard of it, is the thing y'all want.
Whatever happened to Julia? Wasn't it supposed to incorporate all these incredible abstractions at the language level and run quickly on GPUs and everything in-between? Is it just lack of adoption or is has it something else?
If you mean Zygote.jl, it's a very ambitious project (like Swift for Tensorflow which has been under development for even longer I believe) with not many people working on it compared to Tensorflow and pytorch. And Pytorch for example only supports the method it decides to overload, while Zygote aims to support everything in the language (including stuff that isn't as obvious like state, IO, control flow in general).…
https://arxiv.org/abs/1907.07587
https://news.ycombinator.com/item?id=20477873
From the abstract:
> We describe Zygote, a Differentiable Programming system that is able to take gradients of general program structures. We implement this system in the Julia programming language. Our system supports almost all language constructs (control flow, recursion, mutation, etc.) and compiles high-performance code without requiring any user intervention or refactoring to stage computations.
Just linking to this for those who haven't seen it.
Anecdotally, I've dumped TensorFlow in favor of PyTorch for almost all new work I'm doing at my organization (industry focused). Biggest gripes with TensorFlow are overly complex APIs, instability from release to release, constantly broken code in Google's repos, and poor documentation. Maybe TF 2.0 will be better, but for me, the PyTorch ship has already sailed, and I am sailing on it.
Granted that PyTorch and TensorFlow both heavily use the same CUDA/cuDNN components under the hood (with TF also having a billion other non-deep learning-centric components included), I think one of the primary reasons that PyTorch is getting such heavy adoption is that it is a Python library first and foremost. There're maybe all of two "surprises" I've encountered in all my time using it, if even (1. Gradients are…
Yup, you make some great points and I couldn't agree more. Very recently, I was looking into training an object-detector for a custom problem with not many training examples. One of the classes (hardest one to train from few examples) was "person". I was able to create a custom detection network for a 3-class problem, load up the COCO pretrained weights for the network, strip out all the other weights at the "head" f…
Jax, for those that haven't heard of it, is the thing y'all want.
My point is researchers using a framework (matlab) does not mean it’s used heavily in industries or even all industries.
Earlier quoted context omitted.
Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to…
> That may be different now with the various flavors of TF (like TF Eager) Unfortunately, if anything I think it's the opposite. The constant creation and deprecation of TF flavors (tf-eager, tf-slim, tf-learn, keras, tf-estimator, tf.contrib [RIP]) has made reading tensorflow code online somewhat disastrous. Everybody, including the TF team, is using a different API and it's difficult to keep all of them straight. I…