Live data from Hacker News

Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

notamonadtutorial.com

41–50 of 117 posts

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#41

There's also RustPython, a Rust implementation of CPython 3.5+: https://news.ycombinator.com/item?id=20686580 > https://github.com/RustPython/RustPython

Why didn't they call it Rython

Or RytOn ;-)

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#43

This post combines pretty much every technology I'm obsessed with right now: Python, Rust, Pandas, Numpy, and LLVM. Yess!!!

Just a word of caution, always obsess with product and customer needs first :) In ML/data science tech first normally won’t end up well

Unless they are just talking about personal projects?

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#44

whats the benefit of rust over julia or C for computation?

The computation here is actually not done in Rust. The Rust code is performing stages of compilation of the original source code, into an intermediate representation that LLVM finishes compiling. The fully compiled code is what does the computation.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#46

This post combines pretty much every technology I'm obsessed with right now: Python, Rust, Pandas, Numpy, and LLVM. Yess!!!

Just a word of caution, always obsess with product and customer needs first :) In ML/data science tech first normally won’t end up well

People aren't all or always working on products or serving customers.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#47

This post combines pretty much every technology I'm obsessed with right now: Python, Rust, Pandas, Numpy, and LLVM. Yess!!!

Just a word of caution, always obsess with product and customer needs first :) In ML/data science tech first normally won’t end up well

It's important to enjoy your work, which is - among other causes - about having right tools. Also, some of us actually get to have some influence over what language we write our projects in.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#48

Very bizarre there is no discussion of numba here, which has been around and used widely for many years, achieves faster speedups than this, and also emits an LLVM IR that is likely a much better starting point for developing a “universal” scientific computing IR than doing yet another thing that further complicates it with fairly needless involvement of Rust. https://numba.pydata.org/

Did you read the article? If you know of Numba works, you know it can't just pick up different functions from sklearn and scipy and do interprocedural optimization (IPO). For Numba to do that, it'd need all functions involved to be written in Numba @jit style, whereas Weld would work directly on the pre-existing functions.

Rust is just a IPO driver of sorts here.

I'm not critizing Numba btw, I use it regularly, but your comment seems a little off here, considering that Weld has different goal in mind.

Re: Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

#50

whats the benefit of rust over julia or C for computation?

Not to sound like a member of the Rust evangelism strike force, but after using Rust for a couple years, I don't have any desire to go back to C - sum types alone are worth the switch to me, not to mention iterators, concurrency story, etc.

C++ has sum types as of several years ago with std::variant.
Post reply on HN