21–30 of 50 posts
- better syntax (because Julia has proper macro/metaprogramming)
- faster
- automatically works with GPU arrays.Earlier quoted context omitted.
Just use C or C++ or fortran or julia or even lua. The issue of "slow loops" is entirely self-inflicted by some languages. It's getting ridiculous to still worry about this shit in 2022. Simple loops can and should be just as fast as vectorized programs. When they are slower, it is 100% due to deliberate decisions by the language dessigners
This is 100% not the case with SIMD.
It doesn't do any automatic optimization of the loops like some of the projects linked in this thread, but, it provides all the tools needed for humans to express the code in a way that a good compiler can turn it into really good code.
I've found einsum to be amazing at consolidating my code into something more readable, particularly for implementing architectures from scratch. Here's a good video that explains why its so good: https://www.youtube.com/watch?v=pkVwUVEHmfI Also check out Lucid Rains Github, who uses it extensively to build transformer architectures from scratch: https://github.com/lucidrains \ * Example: https://github.com/lucidrains…
Readable if you're already familiar with einsum notation. Otherwise there's learning curve. An alternative to einsum is using multiple dot product and reshape ops, hopefully with each one having a comment - this would be a lot more readable imo.
I've found einsum to be amazing at consolidating my code into something more readable, particularly for implementing architectures from scratch. Here's a good video that explains why its so good: https://www.youtube.com/watch?v=pkVwUVEHmfI Also check out Lucid Rains Github, who uses it extensively to build transformer architectures from scratch: https://github.com/lucidrains \ * Example: https://github.com/lucidrains…
amazing at consolidating my code into something more readable Readable if you're already familiar with einsum notation. Otherwise there's learning curve. An alternative to einsum is using multiple dot product and reshape ops, hopefully with each one having a comment - this would be a lot more readable imo.
https://einops.rocks/pytorch-examples.html shows how it can be used to implement various neural network architectures in a more simplified manor.
I'd really like to use einsum more often, because it allows me to code my expressions the same way I derive them on pen and paper. Unfortunately, as mentioned in the article, it's slow, because it converts your formula to a for loop. So usually, I rewrite my formulas into messy combinations of broadcasts, transposes and array multiplications. Is there a package or an algorithm that does this conversion automatically?…
The Tullio library in Julia is a pretty fantastic option for Einstein summation. It’s performance is great, it generates CUDA kernels, and does some clever tricks for automatic differentiation. It’s also a bit more readable than numpy’s einsum function, since you just write: @tullio C[i,j] := A[i,k] * B[k,j]
Also see Einops: https://github.com/arogozhnikov/einops , which uses a einsum-like notation for various tensor operations used in deep learning. https://einops.rocks/pytorch-examples.html shows how it can be used to implement various neural network architectures in a more simplified manor.
I'd really like to use einsum more often, because it allows me to code my expressions the same way I derive them on pen and paper. Unfortunately, as mentioned in the article, it's slow, because it converts your formula to a for loop. So usually, I rewrite my formulas into messy combinations of broadcasts, transposes and array multiplications. Is there a package or an algorithm that does this conversion automatically?…
The Tullio library in Julia is a pretty fantastic option for Einstein summation. It’s performance is great, it generates CUDA kernels, and does some clever tricks for automatic differentiation. It’s also a bit more readable than numpy’s einsum function, since you just write: @tullio C[i,j] := A[i,k] * B[k,j]