21–30 of 93 posts
I love this! That talk.wasm package has a lot of potential…
So many different cuda version, with each framework using its own, that all rely on a different driver, and everything needs a new version every 3 months and takes ~10G, (and don't even talk about cudnn needing some manual logged-in install).
Here everything is just two files. For embedded system that don't have a GPU it's perfect.
Here the parallelization and vectorization has been done by hand, but there is a glimmer of hope coming from the side of various compiler projects :
Here is an interesting intel project that does the parallelization and vectorization automatically for different architecture that's definitely worth a look : https://ispc.github.io/ispc.html
For the auto-differentiation when I need performance or memory, I currently use tapenade ( http://tapenade.inria.fr:8080/tapenade/index.jsp ) and/or manually written gradient when I need to fuse some kernel, but Enzyme ( https://enzyme.mit.edu/ ) is also very promising.
MPI for parallelization across machines.
Earlier quoted context omitted.
Thanks! Indeed, I agree that the project has an educational aspect and value. For me, it helped me get a better understanding of the neural network layers involved in the transformer model. Also, it was a good playground to practice my low-level optimization techniques. I guess another cool thing was that with the help of the community, we came up with a faster way to evaluate the Encoder (at the cost of some accurac…
I liked reading the different implementations of the low-level tensor ops (simple C/AVX/AVX2/WASM128bit/ARM-NEON) -- it will help me learn about how to use x86 ASM. Thank you for writing this! Do you have any other recommendations/examples on how numerical code can be optimized via SIMD routines?
It looks like it will open a new application scenario for low-performance hardware products.
I looks like you’ve definitely maximized the parallelization on the CPU/AMX here, but have you tried getting it to run on the GPU or the Neural Engine? I love the portability, but I feel like you would get a massive parallelization boost while dramatically cutting energy consumption.
Why not Rust? C++ is obsolete
I wonder how ChatGPT3 would respond to this? If they crawled /r/programming and hackernews, it should be annoyed and come up with something along the lines of "fork it and write in in Rust yourself", "C++ has lots of inertia", "his main stack is C/C++", etc
This is really cool, and I have been meaning to get my hands dirty with Whisper I looks like you’ve definitely maximized the parallelization on the CPU/AMX here, but have you tried getting it to run on the GPU or the Neural Engine? I love the portability, but I feel like you would get a massive parallelization boost while dramatically cutting energy consumption.
Honestly, I am bit confused with all the different types of processing units available on Apple Silicon. If I understand correctly, we have: CPU, GPU, AMX coprocessor and Neural Engine on a single chip. I don't fully understand how these interact with each other. Can we use them all at the same time, or would there be some penalties? I'm interested in finding some resources/information on the topic.