51–60 of 133 posts
...and that's why you are using Keras instead.
I like Keras, but I always find myself having to write TF code whenever I need to implement something more interesting. And debugging, already hard in pure TF, is more complex due to the extra layer. IMO, learning TF or pytorch is more effective at least in the current state of affairs.
An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." There are many people for whom the declarative paradigm is a huge plus. I would say there are at least 2 major approaches in running fast neural networks: 1. Figure out the common big components and make fast versions of those. 2. Figure out the common small components a…
>An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." Cannibalism rules!
Seriously though, the quote works well for many usual cases of discussion. People have incentives which they do believe in - you can discuss how those beliefs started and why you think they don't apply. Sure, you can find pathological edge cases where doing that didn't make sense. Doesn't mean the rule is bad for almost everything else.
An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." There are many people for whom the declarative paradigm is a huge plus. I would say there are at least 2 major approaches in running fast neural networks: 1. Figure out the common big components and make fast versions of those. 2. Figure out the common small components a…
At least Tensorflow isn't at that level, because its "declarative" syntax is just yet another imperative language living on top of Python. But it still makes performance debugging really hard.
With PyTorch, I can just sprinkle torch.cuda.synchronize() liberally and the code will tell me exactly which CUDA kernel calls are consuming how much milliseconds. With Tensorflow, I have no idea why it is slow, or whether it can be any faster at all.
> Pytorch’s interface is objectively much better than Tensorflow’s
How does your subjective opinion about Tensorflow suddenly become objective? I don't see any fact based or metrics based argument that establishes this objectively. All I see is an argument based on your needs and preference. That is far from objective.
Earlier quoted context omitted.
In my experience (computer vision, deep learning) PyTorch is substantially faster as well, especially in data augmentation where it’s not just a thin layer over cudnn. That said, you’re right. There’s no way I’d deploy it to production.
Nonsense, Pytorch is great in production. I use it for Mathpix (mathpix.com) which processes 20 million images per month.
I am not saying that Pytorch is bad in production but I fail to see how your metric of 8 images per second proves anything.
I'm not an AI expert and barely understand how Deep Learning works. It was always my impression that I am the target audience for Tensorflow: mainstream tech workers from the corporate environment who want to add deep learning into their toolbox, while people actually doing research on the matter seemed to prefer Scikit. Wouldn't this justify the adoption of the declarative model and the aggressive abstraction of int…
I think the user-unfriendly of tf mainly comes from these two aspects.
1. The choice of python.
When I'm writing python, I spent more time debugging, (compared to something like OCaml), and sometimes it takes more time to get started because arguments in functions are documented rather than enforced by contract/compiler, (and those "if this pass arg1 else pass arg2" documentation will never let you have the same kind confidence as you would if you are using a more rigorous language), so you end up trying. This isn't specific to tf, but tf makes this more obvious because it's something like a language-in-language.
If tf was made by someone else, I would understand the choice, because the popularity of python and tons of library available, but since Google has unlimited resource, and AI is clearly the future, I really expect they have the courage to choose something else.
Sometimes I wonder, AI may gone rouge some day - not because that we deliberately make it so, but that a bug somewhere in our code.
2. Lack of maintenance.
We all like shiny new ideas, and get excited implementing them, but once the fun part is done, so goes the excitement. A good library needs to be tweaked and re-tweaked, some of these need boring hard work, smart people don't like that.
But hey, Google is doing this for free, as long as it's not deliberately made so (to stall the community), we should be appreciate, it's a open source project and that don't just mean we can use it for free, but also that we should done our part to make it better.
I'm not an AI expert and barely understand how Deep Learning works. It was always my impression that I am the target audience for Tensorflow: mainstream tech workers from the corporate environment who want to add deep learning into their toolbox, while people actually doing research on the matter seemed to prefer Scikit. Wouldn't this justify the adoption of the declarative model and the aggressive abstraction of int…