I think the author raises a good point about Google envy. TensorFlow is not the most intuitive or flexible library out there, and it is very over-engineered if you're not doing large-scale distributed training. The main reason why everyone talks it up so much is because Google heavily marketed it from the outset, and everyone automatically assumes Google == Virtuoso Software Design because they couldn't make it through the interview. Really it's just modern enterprise software which has five different ways to implement batch norm that they push on the community so they don't have to train new hires on how to use it.
Tensorflow sucks
41–50 of 133 posts
Re: Tensorflow sucks
#42An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." There are many people for whom the declarative paradigm is a huge plus. I would say there are at least 2 major approaches in running fast neural networks: 1. Figure out the common big components and make fast versions of those. 2. Figure out the common small components a…
>An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." Cannibalism rules!
Lmao not sure why this was down-voted. Utterly destroyed your quotation, and gave me a giggle.
Re: Tensorflow sucks
#43Earlier quoted context omitted.
Because TF lets you run _just_ the model, with no Python stuff at all. For us there is no “server side”, our stuff is deployed on prem. The fewer variables there are the easier our lives are going to be. Additionally, PyTorch download page warns you point blank that it’s an early version of the software and that you should “expect some adventures”. Adventures are fine for research, but inadvisable in production IMO.
I understand the appeal, but IMO wrapping the Python stuff in a docker container is fairly simple. Making it run at scale using something like kubernetes is more advanced stuff but still within good devops practices. Unless, of course if you want to run on exotic HW, like TPUs. But that's an issue for Googlish scales. Edit: BTW, if on premise you mean Windows client software your point is totally valid, Python would…
Depending on how much extra logic you add around the model serving library, then you are exposing your source code, too. Maybe you care, maybe not.
Re: Windows, there's no TF serving for that, so TF is no better than other Python frameworks.
Re: Tensorflow sucks
#44A few more for your selection, - Bloated build system that is near impossible to get working - who even uses maven ?! Pytorch/Caffe are super-simple to build in comparison; with Chainer, it's even simple: all you need is pip install (even on exotic ARM devices). - The benefits of all that static analysis simply aren't there. In addition, PyTorch has a jit-compiler which one can argue lets one have their cake and eat…
I love that PyTorch kind of went all-in with anaconda. Building it is so much easier than TF! I'm a recent convert but it's dang good.
Why not pip?
Re: Tensorflow sucks
#45Hell, being able to effortlessly switch between PyTorch and Numpy/SciPy/sklearn/skimage has been so helpful for the project I'm working on. That and I have tensors in later layers whose shapes depend on the training of the previous layers.
I have tensors in later layers whose shapes depend on the training of the previous layers.
Rad! Do you have any examples (or literature) that explains when this is beneficial?
Re: Tensorflow sucks
#46Despite its shortcomings, I share the same vision as this article. Here are my reasons:
- Tensorflow has a way too large API surface area: parsing command lines arguments handling, unit test runners, logging, help formatting strings... most of those are not as good as available counterparts in python.
- The C++ and Go versions are radically different from the Python version. Limited code reuse, different APIs, not maintained or documented with the same attention.
- The technical debt in the source code is huge. For instance, There are 3 redundant implementations in the source code of a safe division (_safe_div), with slightly different interfaces (sometimes with default params, sometimes not). It's technical debt.
In every way, it reminds me of Angular.io project. A failed promise to be true multi-language, failing to use the expressiveness of python, with a super large API that tries to do things we didn't ask it to do and a lack of a general sounding architecture.
Re: Tensorflow sucks
#47Hell, being able to effortlessly switch between PyTorch and Numpy/SciPy/sklearn/skimage has been so helpful for the project I'm working on. That and I have tensors in later layers whose shapes depend on the training of the previous layers.
I have tensors in later layers whose shapes depend on the training of the previous layers. Rad! Do you have any examples (or literature) that explains when this is beneficial?
Not yet! I'm not using convnets or backprop or anything so I don't think it would be beneficial that way, but you could get something similar to what I'm doing by looking at Fritzke's Growing Neural Gas[1]
[1] http://papers.nips.cc/paper/893-a-growing-neural-gas-network...
Re: Tensorflow sucks
#48Earlier quoted context omitted.
I love that PyTorch kind of went all-in with anaconda. Building it is so much easier than TF! I'm a recent convert but it's dang good.
Why not pip?
For installing it, yeah pip is great too, but for building conda includes third party tools and libraries and stuff. e.g. in order to use the MPI backend for PyTorch's distributed processing you need to build it yourself and conda just makes it a bit easier. That and I had a real bad experience with trying to build Tensorflow (and Bazel) to run on an HPC cluster.
Re: Tensorflow sucks
#49This article is not that detailed, but it's a sentiment I agree with, so I'll add one major shortcoming of Tensorflow: its memory usage is really bad.
The default behavior of TF is to allocate as much GPU memory as possible for itself from the outset. There is an option (allow_growth) to only incrementally allocate memory but when I tried it recently it was broken. This means there aren't easy ways to figure out exactly how much memory TF is using (e.g. if you want to increase the batch size). I believe you can use their undocumented profiler, but I ended up just tweaking batch sizes until TF stopped crashing (yikes).
TF does not have in-place operation support for some common operations that could use it, like dropout (other operations do have this support, I believe). Even Caffe, which I used for my research in college, had this. This can double your GPU RAM usage depending on your model, and GPU RAM is absolutely a precious resource.
Finally, I've had issues where TF runs out of GPU RAM halfway through training, which should never happen - if there's enough memory for the first epoch, there should be enough memory for every epoch. The last thing I want to do is debug a memory leak / bad memory allocation ordering in TF.
Re: Tensorflow sucks
#50How do you even make a "Tensorflow Sucks" article and not mention CNTK?