Making Python faster with Rust
ohadravid.github.io
Making Python faster with Rust
1–10 of 223 posts
Re: Making Python faster with Rust
#2Making Python (near infinitely) faster by using it as a glue language, and running all the computation outside Python :-P
Re: Making Python faster with Rust
#3Python and Rust. Apple has some job openings in distributed systems for people fluent in both.
Re: Making Python faster with Rust
#4> The library was already using numpy for a lot of its calculations, so why should we expect Rust to be better?
I literally clicked in to read the article to see if they'd mention this:) But... unless I missed it, there wasn't really an answer? I thought numpy does do the heavy lifting in native code, so why is this faster? Does this version just push more of the logic into native code than numpy did?
Re: Making Python faster with Rust
#5Python and Rust. Apple has some job openings in distributed systems for people fluent in both.
Is Apple still hybrid only, no remote? Any links to some of their careers pages on this?
Piqued my curiosity as my background almost lines up, and I'm interested in that kind of role... But I live near one of their offices only hiring for skill sets I don't really have.
Re: Making Python faster with Rust
#6Very cool! I can see myself using this soon actually :) On top of the "code speed up" this is a good problem for 2d data structures for performing this type of "find objects within radius" type of query.
Re: Making Python faster with Rust
#7> The library was already using numpy for a lot of its calculations, so why should we expect Rust to be better? I literally clicked in to read the article to see if they'd mention this:) But... unless I missed it, there wasn't really an answer? I thought numpy does do the heavy lifting in native code, so why is this faster? Does this version just push more of the logic into native code than numpy did?
People can write slow code in any language ;-) We've had to fire contractors who "wrote" dataframe code but was not vectorized in practice despite repeat request to do so. Same thing for slow code in CUDA.
From a maintenance view, I much prefer folks write vectorized data frames vs numpy or low level bindings, but that comes from having lived with the alternative for a lot longer. All of our exceptions are pretty much slotted for deletion. (Our fast path is single or multi-GPU dataframes in python.) Here's to hoping that one day we'll have dependent types in mypy!
Re: Making Python faster with Rust
#8This is possibly one of the best written articles end-to-end I have read. Excellent job telling the story
Re: Making Python faster with Rust
#9> The library was already using numpy for a lot of its calculations, so why should we expect Rust to be better? I literally clicked in to read the article to see if they'd mention this:) But... unless I missed it, there wasn't really an answer? I thought numpy does do the heavy lifting in native code, so why is this faster? Does this version just push more of the logic into native code than numpy did?
The slowness comes from the interaction of numpy and a Python object "Polygon", which in not numpy. I suspect that a sufficiently clever coder could have optimized the result without resorting to Rust, but at the cost of a substantial increase in complexity of the codebase. The proposed approach keep the Python code simple (and moves the complexity into having another language to deal with).
Re: Making Python faster with Rust
#10Good for you! You did everything right: measure always, fix the bottleneck if possible, rewrite if necessary.
A little tip, you don't have to compare actual distances, you can compare squared distances just as well. Then in `norm I once rewrote a GDI+ point transformation routine in pure C# and got 200x speedup just because the routine was riddled with needless virtual constructors, copying type conversions, and something called CreateInstanceSlow. Ten years after, I gathered a few of these anecdotes and wrote the Geometry for Programmers book (https://www.manning.com/books/geometry-for-programmers) with its main message: when you know geometry behind your tools, you can either use them efficiently, or rewrite them completely.