Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Calling Rust from Python using PyO3

saidvandeklundert.net

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Calling Rust from Python using PyO3

#31

Post by Jansen312 »

wheelerof4te wrote: post #18
linkdd wrote: post #13

Earlier quoted context omitted.

While Python's bindings in other languages are great, and Rust has amazing features regarding memory safety, I think saying they are the best in their area is a bit too much. For compiled languages: Rust is the best when you care about memory safety. C is the best when you care about performance/simplicity/portability. C++ is the best when you care about modularity, Assembly is the best when you care about lowlevel s…

I said that Rust "is on track" to become the best, not that it is best. It is still a young language, but Rust shows great promise to one day be that one language you need for just about anything, sans scripting. The best part is that Rust is approaching C levels of speed, sometimes even surpassing it. Python is currently the easiest language to learn, has a plethoa of modules in it's standard library alone, plus a v…

Compiled with memory safety? Which language you have in mind better than rust at this moment? Just curious.

Re: Calling Rust from Python using PyO3

#32

Post by wheelerof4te »

Jansen312 wrote: post #31
wheelerof4te wrote: post #18

Earlier quoted context omitted.

I said that Rust "is on track" to become the best, not that it is best. It is still a young language, but Rust shows great promise to one day be that one language you need for just about anything, sans scripting. The best part is that Rust is approaching C levels of speed, sometimes even surpassing it. Python is currently the easiest language to learn, has a plethoa of modules in it's standard library alone, plus a v…

Compiled with memory safety? Which language you have in mind better than rust at this moment? Just curious.

Honestly, none that I can think of.

Re: Calling Rust from Python using PyO3

#33

Post by HelloNurse »

dijit wrote: post #10
gravypod wrote: post #2

I hope someone creates a rule set for Bazel to automate this stuff. This seems like a really nice way to start rewriting performance critical code in a safe language!

I’ve been having difficulty recently trying to get bazel and rust to work together nicely. It seems like cargo does a lot of heavy lifting w.r.t. dependencies which Bazel does not like. Do you have to vendor your dependencies- and your dependencies dependencies. Ad infinitum. There is cargo-raze which helps, but only if you’re making a rust library: not if you’re making a binary. So maybe it works for this case.

Isn't it possible to make a little script to run bazel clean or the equivalent whenever you run Cargo to actually add, remove or update Rust libraries, and ignore them as dependencies because they are immutable apart from such user interventions? Are Bazel and/or Cargo too smart?

Re: Calling Rust from Python using PyO3

#34

Post by adsharma »

tialaramex wrote: post #29
linkdd wrote: post #13

Earlier quoted context omitted.

While Python's bindings in other languages are great, and Rust has amazing features regarding memory safety, I think saying they are the best in their area is a bit too much. For compiled languages: Rust is the best when you care about memory safety. C is the best when you care about performance/simplicity/portability. C++ is the best when you care about modularity, Assembly is the best when you care about lowlevel s…

Other people have pushed back about safety, but I want to push back on performance . Here's the thing, presumably you agree that "It's fast except that it never works" isn't actually fast. So in practice your high performance C ends up compromised by the reality that it must work, at least often, which constrains what you are confident to actually write because you already know you can't write over-complicated code c…

Agreed on the need to write simpler programs with less cognitive load.

The thing about wuffs is that you can't single step through it and there is no ecosystem of libraries a wuffs programmer could use.

This is why a subset of python that shares design principles with Julia and Nim, but transpiles to Rust, Go or C++ is interesting. It's not hugely popular with those language communities (prefer coding natively in Julia or Nim), but the pytorch thread (https://news.ycombinator.com/item?id=29354474#29371641) explains why the ecosystem is important (harder to build).

Having said that the basic hello world wuffs example in py2many isn't working as well as I'd like it to, but it's close.

Re: Calling Rust from Python using PyO3

#37

Post by vlmutolo »

We used PyO3 at $work to expose a Rust implementation of a compute-intensive algorithm to an existing Python codebase.

The teammate who did it had been using Rust only for a couple months and none of us had ever used PyO3. He got it done in just a couple days. I consider that an endorsement of the API they've built.

It's heavily macro-based, which does cause some confusion. But if you spend some time with their examples, finding the fast path isn't too tricky.

Re: Calling Rust from Python using PyO3

#38

Post by staticassertion »

vlmutolo wrote: post #37

We used PyO3 at $work to expose a Rust implementation of a compute-intensive algorithm to an existing Python codebase. The teammate who did it had been using Rust only for a couple months and none of us had ever used PyO3. He got it done in just a couple days. I consider that an endorsement of the API they've built. It's heavily macro-based, which does cause some confusion. But if you spend some time with their examp…

Did you run into any issues? I've been nervous about taking this approach in our codebase because it doesn't feel totally well worn yet, but anecdotes are exactly the sort of thing that change my mind about that.

Re: Calling Rust from Python using PyO3

#39

Post by staticassertion »

tialaramex wrote: post #29
linkdd wrote: post #13

Earlier quoted context omitted.

While Python's bindings in other languages are great, and Rust has amazing features regarding memory safety, I think saying they are the best in their area is a bit too much. For compiled languages: Rust is the best when you care about memory safety. C is the best when you care about performance/simplicity/portability. C++ is the best when you care about modularity, Assembly is the best when you care about lowlevel s…

Other people have pushed back about safety, but I want to push back on performance . Here's the thing, presumably you agree that "It's fast except that it never works" isn't actually fast. So in practice your high performance C ends up compromised by the reality that it must work, at least often, which constrains what you are confident to actually write because you already know you can't write over-complicated code c…

There's a lot of examples of this sort of thing.

1. C++ uses the equivalent of an Arc in Rust, because it can't tell if that reference will be shared across threads. In Rust you can use an Rc and, if you ever need an Arc, it will tell you.

2. Rust's `&str[..]` is safe, C++'s string_view causes tons of UAFs, so string_view is used much less frequently whereas &str is ubiquitous in rust code. In general you can share stack space in Rust easily, even across threads, which is incredibly powerful.

Re: Calling Rust from Python using PyO3

#40

Post by jamesmishra »

staticassertion wrote: post #35

I'm curious about any rough edges. 1. What happens if the Rust code is misbehaving, panicking, etc? 2. What's build support like for this? I'd love to gut what little Python code I have left and use Rust.

1. The Python code calling the Rust code raises a runtime-related exception, but it doesn't crash the process.

2. Building Python extensions out of PyO3 code is easy with maturin: https://github.com/PyO3/maturin

Post reply on HN