Live data from Hacker News

Calling Rust from Python using PyO3

saidvandeklundert.net

41–50 of 51 posts

Re: Calling Rust from Python using PyO3

#41

This is smart thinking. Python is the world's best glue language and Rust is on track to become the world's best compiled, safe language. It makes sense to combine the two and replace C in this.

C has not been fully replaced. These Python modules are not Rust libraries, they're C libraries that just happen to be written in Rust internally.

Re: Calling Rust from Python using PyO3

#42

We used PyO3 at $work to expose a Rust implementation of a compute-intensive algorithm to an existing Python codebase. The teammate who did it had been using Rust only for a couple months and none of us had ever used PyO3. He got it done in just a couple days. I consider that an endorsement of the API they've built. It's heavily macro-based, which does cause some confusion. But if you spend some time with their examp…

Did you run into any issues? I've been nervous about taking this approach in our codebase because it doesn't feel totally well worn yet, but anecdotes are exactly the sort of thing that change my mind about that.

No real issues other than our own lack of experience with PyO3. I'd say the biggest gamble is spending the time to get it set up. Once we got it running, it was smooth sailing.

We're calling Rust from Python. Haven't tried it the other way around. Our use case was helped by the fact that we are just passing a String into Rust and letting Rust do all the heavy lifting. There's minimal back-and-forth.

I liked how PyO3 managed panics (they're just normal exceptions that can be caught on the Python side). I wasn't the one dealing with Maturin, but it seemed reasonable to get started with. I never enjoy introducing more tools into a build system, but this was relatively painless.

If I recall correctly, the bindings themselves are only like… thirty lines of code.

Re: Calling Rust from Python using PyO3

#43

This is smart thinking. Python is the world's best glue language and Rust is on track to become the world's best compiled, safe language. It makes sense to combine the two and replace C in this.

C has not been fully replaced. These Python modules are not Rust libraries, they're C libraries that just happen to be written in Rust internally.

By "C" libraries you just mean that python and rust talk to eachother via the same ABI that C happens to use, there doesn't need to actually be any C involved. Since python is generally the cPython implementation of python there is, but you could substitute that out for something else (i.e. a python implementation written in Java) and actually have no C without modifying anything else.

Well, except for the fact that the JVM and OS is written in C, but you could swap out that JVM and OS for...

Re: Calling Rust from Python using PyO3

#45

This is smart thinking. Python is the world's best glue language and Rust is on track to become the world's best compiled, safe language. It makes sense to combine the two and replace C in this.

C has not been fully replaced. These Python modules are not Rust libraries, they're C libraries that just happen to be written in Rust internally.

Rust uses the C ABI to interface with foreign code, including code that happens to be written in Rust but may have been built with a different toolchain release. There are crates to make this easier even when using non-trivial Rust features that don't have an obvious C equivalent.

Re: Calling Rust from Python using PyO3

#46
post #14

Earlier quoted context omitted.

C, C++, and Assembly aren't safe, and Java and JS aren't good glue languages.

Implementing Rust's borrow checker as a C++ compile-time template metaprogramming exercise seems only a guru and a cloud-based compiler cluster out of reach.

Rust's borrow checker works perfectly well written in Rust without needing a "cloud-based compiler cluster".

But I'm guessing what you meant to suppose here would be instead to do this for C++ and that's not going to happen, though it does for some reason seem to be a popular wish of C++ enthusiasts. C++ doesn't express the thing the borrow checker is er... checking. So you'd need to adjust the language, Stroustrup has some sketched proposals about that, but realistically they're years out even if the C++ community was enthusiastic which they are not.

Re: Calling Rust from Python using PyO3

#48

Earlier quoted context omitted.

Implementing Rust's borrow checker as a C++ compile-time template metaprogramming exercise seems only a guru and a cloud-based compiler cluster out of reach.

Rust's borrow checker works perfectly well written in Rust without needing a "cloud-based compiler cluster". But I'm guessing what you meant to suppose here would be instead to do this for C++ and that's not going to happen, though it does for some reason seem to be a popular wish of C++ enthusiasts. C++ doesn't express the thing the borrow checker is er... checking. So you'd need to adjust the language, Stroustrup h…

It was a semi-tongue-in-cheek observation along the lines of => https://abel.sinkovics.hu/download/metamonad.pdf

Re: Calling Rust from Python using PyO3

#50

Earlier quoted context omitted.

Other people have pushed back about safety, but I want to push back on performance . Here's the thing, presumably you agree that "It's fast except that it never works" isn't actually fast. So in practice your high performance C ends up compromised by the reality that it must work, at least often, which constrains what you are confident to actually write because you already know you can't write over-complicated code c…

There's a lot of examples of this sort of thing. 1. C++ uses the equivalent of an Arc in Rust, because it can't tell if that reference will be shared across threads. In Rust you can use an Rc and, if you ever need an Arc, it will tell you. 2. Rust's `&str[..]` is safe, C++'s string_view causes tons of UAFs, so string_view is used much less frequently whereas &str is ubiquitous in rust code. In general you can share s…

I would guess that Arc ends up costing nothing on Intel. Intel's aligned load/ stores come with Acquire/Release atomic semantics, so even if you don't need them you don't get a discount. On modern ARM platforms they're very cheap but perhaps not quite free, and realistically there just aren't that many other platforms. So the Arc/Rc difference may make little practical difference.

I think the clear benefit str has is that it's built in from day one. So any code which doesn't use str but should isn't old code it's just bad code. In contrast if you've got a pile of C++ then nothing written before 2017 uses string_view because string_view did not exist, and so then much of what got written after 2017 also didn't use string_view because the old code didn't understand string_view.

Post reply on HN