61–70 of 73 posts
Earlier quoted context omitted.
It sounds like you've found something that works best for you, and that the large Keras user base has found something that works best for them.
The large Keras userbase exists largely because Tensorflow sucked.
It's also a bit histrionic, in that Keras was very popular with the Theano backend before that project wound down.
Earlier quoted context omitted.
Pytorch is an animal by itself when you try to put it into production. They have started addressing it with torch 2.0 but it still has lengths to go. With this you can switch to TFserve if you have usual architecture.
You can just use Triton which is basically TFserve for Tensorflow, Pytorch, Onnx and more.
My understand of Triton is more that this is an alternative to CUDA, but instead you write it directly in Python, and on a slightly higher-level, and it does a lot of optimizations automatically. So basically: Python -> Triton-IR -> LLVM-IR -> PTX.
Can someone ELI5 the relationship between Keras and TensorFlow/Jax/PyTorch/etc? I kinda get the idea the Keras is the "frontend" and TF/Jax/PyTorch are the "backend" but I'm looking to solidify my understanding of the relationship. It might help to also comment on the key differences between TF/Jax/PyTorch/etc. Thank you.
The creator of Keras was then employed by Google to work on Keras, who promised everyone Keras would remain backend agnostic.
Keras become part of Tensorflow as a high-level API and did not remain backend agnostic. There was lots of questionable twitter beef about Pytorch by the Keras creator.
Keras is now once again backend agnostic, as a high-level API for Tensorflow/PyTorch/Jax. Likely as them seeing Tensorflow losing traction.
Earlier quoted context omitted.
You can just use Triton which is basically TFserve for Tensorflow, Pytorch, Onnx and more.
Can you explain that? My understand of Triton is more that this is an alternative to CUDA, but instead you write it directly in Python, and on a slightly higher-level, and it does a lot of optimizations automatically. So basically: Python -> Triton-IR -> LLVM-IR -> PTX. https://openai.com/research/triton
Supporting multiple backends (especially Jax) is nice! Makes experimenting/migrating between them so much more approachable. Any timeline on when can we expect support for distributed Jax training? The doc currently seems to indicate only TF is supported for distributed training.
Earlier quoted context omitted.
How so? You can use torch.nn.Sequential pretty much equivalently? https://pytorch.org/docs/stable/generated/torch.nn.Sequentia...
Huh, didn't realize base PyTorch had an equivalent Sequential API. The point about the better API overall still stands (notably including the actual training part, as base PyTorch requires you to implement your own loop)
Keras has always been "lipstick on a pig for tensorflow". Its value beyond that seems tenuous at best.
Earlier quoted context omitted.
The same reason why you might want to use Keras if you use any of the other backends. They operate at different levels. Keras is a higher-level API. It means that you can prototype architectures quickly and you don't have to write a training loop. It's also really easy to extend. I currently use PyTorch Lightning to avoid having to write tonnes of boilerplate code, but I've been looking for a way to leave it for ages…
What’s so hard about writing a training loop?
Earlier quoted context omitted.
Can you explain that? My understand of Triton is more that this is an alternative to CUDA, but instead you write it directly in Python, and on a slightly higher-level, and it does a lot of optimizations automatically. So basically: Python -> Triton-IR -> LLVM-IR -> PTX. https://openai.com/research/triton
It's confusing, there's OpenAI Triton (what you're thinking of) and Nvidia Triton server (a different thing).