I didn't understand what was being said here; was he suggesting that you call libc free using FFI; which would be fine? I understand the interviewer asked about using Rust dealloc though. I think the FFI bit is confusing me.
A deep dive into Rust and C memory interoperability
61–70 of 84 posts
Re: A deep dive into Rust and C memory interoperability
#62The reason you are not seeing crashes when allocating with Rust and freeing with C (or vice versa) is that by default Rust also uses the libc allocator. https://stdrs.dev/nightly/x86_64-unknown-linux-gnu/src/std/s...
Miri and Valgrind will usually catch this kind of issue. I did lots of work mixing C and Rust and that tripped me as well at the beginning. I wrote about it if someone is interested: https://gaultier.github.io/blog/perhaps_rust_needs_defer.htm...
1. An object that’s really a Rust Vec under the hood is awkward when you try to pass ownership to C. ISTM you end up solving it by creating what is, in effect, a handle, and freeing that using a custom free function. Is this really a problem with Rust FFI? This style of API is very common in native C code, and it’s a lot more flexible and future-proof than having a library user call free() directly.
2. You’re consuming the API in Rust and being sad that there’s no native defer statement. But Rust can do RAII — could you build a little wrapper struct that implements Drop?
Re: A deep dive into Rust and C memory interoperability
#63Earlier quoted context omitted.
While I usually hate all the accusations of writings being LLM generated, I find your example a bit odd as that phasing is very typical of ChatGPT, especially when it was glazing everyone after that one update they had to reverent. “It’s not just _________. It’s _________________.” This was in almost every response doubling down on the users ideas and blowing things out of proportion. Stuff like… “It’s not just a goo…
huh, interesting, I guess I haven't read enough of it to pick up on the patterns
He's not the only one to point out these things that LLMs (currently) tend to output, but this is one of the shorter overviews of the tells you can spot.
Re: A deep dive into Rust and C memory interoperability
#64The reason you are not seeing crashes when allocating with Rust and freeing with C (or vice versa) is that by default Rust also uses the libc allocator. https://stdrs.dev/nightly/x86_64-unknown-linux-gnu/src/std/s...
It's funny. When I first tried Rust in 2018 they were still statically linking jemalloc into every binary rustc compiled by default, and that alone very much put me off of the language for a while. Apparently they did away with jemalloc in favor of the system allocator that same year but nonetheless when I came back to it years later I was very happy to learn of its removal.
Re: A deep dive into Rust and C memory interoperability
#65Earlier quoted context omitted.
While I usually hate all the accusations of writings being LLM generated, I find your example a bit odd as that phasing is very typical of ChatGPT, especially when it was glazing everyone after that one update they had to reverent. “It’s not just _________. It’s _________________.” This was in almost every response doubling down on the users ideas and blowing things out of proportion. Stuff like… “It’s not just a goo…
huh, interesting, I guess I haven't read enough of it to pick up on the patterns
Re: A deep dive into Rust and C memory interoperability
#66Earlier quoted context omitted.
That seems suspect to me. If I call reserve_exact I do actually mean reserve_exact and I want .capacity() to return with the argument I passed to reserve_exact(). This is commonly done when using Vec as a fixed capacity buffer and you don't want to add another field to whatever owns it that's semantically equivalent to .capacity(). I don't really care if the memory region is past capacity * size of:: (), but I do wan…
> This is commonly done when using Vec as a fixed capacity buffer and you don't want to add another field to whatever owns it that's semantically equivalent to .capacity(). The documentation for Vec already explains exactly what it's offering you, but lets explore, what exactly is the problem? You've said this is "commonly done" so doubtless you can point at examples for reference. Suppose a Goose is 40 bytes in size…
> Now, what disaster awaits in the common code you're talking about? Capacity is 6 and... there's capacity for 6 entries instead of 4
The capacity was expected to be 4 and not 6, which may be a logical error in code that requires it to be. If this wasn't a problem the docs wouldn't call it out as a potential problem.
Re: A deep dive into Rust and C memory interoperability
#67Earlier quoted context omitted.
Jemalloc added over a megabyte to every project for only questionable gains, and it was awkward and unwieldy to remove it. While there are good reasons to use a different allocator depending on the project, Rust defaulting to this type of behavior failed a certain personal litmus test on what it wanted to be as a language in that it felt like it was fighting the system rather than integrating with it. It also does no…
> Jemalloc added over a megabyte to every project Right, but, so what? I can't imagine a practical reason this matters for the vast majority of situations. So either you're doing something pretty niche, or it's a purely aesthetic complaint.
Re: A deep dive into Rust and C memory interoperability
#68Earlier quoted context omitted.
Jemalloc added over a megabyte to every project for only questionable gains, and it was awkward and unwieldy to remove it. While there are good reasons to use a different allocator depending on the project, Rust defaulting to this type of behavior failed a certain personal litmus test on what it wanted to be as a language in that it felt like it was fighting the system rather than integrating with it. It also does no…
> Jemalloc added over a megabyte to every project Right, but, so what? I can't imagine a practical reason this matters for the vast majority of situations. So either you're doing something pretty niche, or it's a purely aesthetic complaint.
Those are certainly valid. There are entire Linux distros whose mere existence is as the answer to what is only an aesthetic complaint.
Re: A deep dive into Rust and C memory interoperability
#69Earlier quoted context omitted.
Jemalloc added over a megabyte to every project for only questionable gains, and it was awkward and unwieldy to remove it. While there are good reasons to use a different allocator depending on the project, Rust defaulting to this type of behavior failed a certain personal litmus test on what it wanted to be as a language in that it felt like it was fighting the system rather than integrating with it. It also does no…
> Jemalloc added over a megabyte to every project Right, but, so what? I can't imagine a practical reason this matters for the vast majority of situations. So either you're doing something pretty niche, or it's a purely aesthetic complaint.
Re: A deep dive into Rust and C memory interoperability
#70One of the areas I wonder about this a lot is when integrating Rust code into Postgres which has its own allocator system. Mostly right now when we need to have complex data structures (non-Postgres data structures) that must live outside of the lexical scope we put them somewhere global and return a handle to the C code to reference the object. But with the upcoming support for passing an allocator to any data struc…
>But with the upcoming support for passing an allocator to any data structure (in the Rust standard library anyway) I think this gets a lot easier? Yes and no. Even within libstd, some things require A=GlobalAlloc, eg `std::io::Read::read_to_end(&mut Vec )` will only accept Vec . It cannot be changed to work with Vec because that change would make it not dyn-compatible (nee "object-safe"). And as you said it will cut…
Not sure if that is a breaking change though, it probably is because of a small detail, I'm not a rustc dev.