Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
notamonadtutorial.com
Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
1–10 of 117 posts
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#2whats the benefit of rust over julia or C for computation?
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#3whats the benefit of rust over julia or C for computation?
from TFA:
>> We chose Rust because:
>> It has a very minimal runtime (essentially just bounds checks on arrays) and is easy to embed into other languages such as Java and Python
>> It contains functional programming paradigms such as pattern matching that make writing code such as pattern matching compiler optimizations easier
>> It has a great community and high quality packages (called “crates” in Rust) that made developing our system easier.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#4See also this interesting talk on Weld at RustConf 2019: https://www.youtube.com/watch?v=AZsgdCEQjFo&t=1430s
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#5whats the benefit of rust over julia or C for computation?
I like the memory safety, concurrency, and zero-based indices.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#6There's also RustPython, a Rust implementation of CPython 3.5+: https://news.ycombinator.com/item?id=20686580
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#7whats the benefit of rust over julia or C for computation?
Rust performance is pretty much in terms of C/C++ performance but promises stability in regards to memory management due to the borrow checker. Rust is very impressive on its own. If you havent taken the time to research Rust because "its yet another language" you really ought to honestly.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#8This project has similar goals to the MLIR project:
https://github.com/tensorflow/mlir
https://www.youtube.com/watch?v=qzljG6DKgic
Exciting times for the future of parallel computing!
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#9I saw a performance comparision with XLA, and it's interesting that Weld is faster, because XLA is supposed to optimize the code using the known tensor sizes during compile time.
Weld and XLA seem to have similar optimization steps though.
Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
#10Very bizarre there is no discussion of numba here, which has been around and used widely for many years, achieves faster speedups than this, and also emits an LLVM IR that is likely a much better starting point for developing a “universal” scientific computing IR than doing yet another thing that further complicates it with fairly needless involvement of Rust.