Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust for Filesystems

lwn.net

Post reply on HN
Print view View original

191–200 of 206 posts

Re: Rust for Filesystems

#191

Post by Smaug123 »

gwbas1c wrote: post #180
Smaug123 wrote: post #156

Earlier quoted context omitted.

The point is that Rust can model invariants that C can't. You can call both ways, but if C is incapable of expressing what Rust can, that has important implications for the design of APIs which must be common to both.

That's not how I interpreted it: There is a clear need to be able to write filesystems in Rust, and the kernel developer(s) who write the filesystem API don't want to have to maintain the bindings to Rust.

They say this in almost every paragraph! For example, five of the first seven paragraphs:

> The first is to express more of the requirements using Rust's type system in order to catch more mistakes at compile time.

> Almeida showed an example of how the Rust type system can eliminate certain kinds of errors.

> … it was exactly that kind of discussion/argument that could be avoided by encapsulating the rules into the Rust types and abstractions; the compiler will know the right thing to do.

> … All of that is enforced through the type system.

> the whole idea is to determine what the constraints are from Viro and other filesystem developers, then to create types and abstractions that can enforce them.

More explicitly:

> The object lifecycles are being encoded into the Rust API, but there is no equivalent of that in C; if someone changes the lifecycle of the object on one side, the other will have bugs.

> As those changes occur, "we will find out whether or not this concept of encoding huge amounts of semantics into the type system is a good thing or a bad thing".

Re: Rust for Filesystems

#192

Post by gwbas1c »

Smaug123 wrote: post #191
gwbas1c wrote: post #180

Earlier quoted context omitted.

That's not how I interpreted it: There is a clear need to be able to write filesystems in Rust, and the kernel developer(s) who write the filesystem API don't want to have to maintain the bindings to Rust.

They say this in almost every paragraph! For example, five of the first seven paragraphs: > The first is to express more of the requirements using Rust's type system in order to catch more mistakes at compile time. > Almeida showed an example of how the Rust type system can eliminate certain kinds of errors. > … it was exactly that kind of discussion/argument that could be avoided by encapsulating the rules into the…

> In addition, when the C code changes, the Rust code needs to follow along, but who is going to do that work? Almeida agreed that it was something that needs to be discussed.

FWIW: I shipped a Windows file system driver in 2020. The api hadn't changed in years. Does Linux's API for kernel-space filesystems really change so rapidly that keeping the rust bindings up-to-date would be a considerable amount of work, in the long run?

Re: Rust for Filesystems

#193

Post by spacechild1 »

LegibleCrimson2 wrote: post #187
spacechild1 wrote: post #184

Earlier quoted context omitted.

I think you're confusing some terms here. > In C++ you need an `extern "C"`, because C++ linkage isn't guaranteed to be the same as C linkage. `extern "C"` has nothing to do with linkage, all it does is disable namemangling, so you get the same symbol name as with a C compiler. > Throwing a C++ static function as a callback into a C function usually works, but it's not technically correct because the linkage isn't gu…

> `extern "C"` has nothing to do with linkage, all it does is disable namemangling, so you get the same symbol name as with a C compiler. extern "C" also ensures that the C calling convention is used, which is relevant for callbacks. It's not just name mangling. This is the reason that extern "C" static functions exist. You can actually overload a C++ function by extern "C" vs extern "C++", and it will dispatch it ap…

> extern "C" also ensures that the C calling convention is used, which is relevant for callbacks. It's not just name mangling.

I stand corrected. I didn't know that `extern "C"` enforces the C calling convention.

However, on modern platforms this doesn't really matter because, as I said, there is only a single calling convention (per platform). And I'm pretty sure that future platforms will keep it that way. Fortunately, if you try to pass a C++ callback of the wrong calling convention, you get a compiler error.

> If you're passing a callback to a C function from C++, it's wrong unless the callback is declared extern "C".

That's certainly not true because `extern "C"` is not the only way to specify the calling convention. In fact, you might need a different calling convention! As I mentioned, on x86 the Windows API uses stdcall for all API functions and callbacks, so `extern "C"` would be wrong. If you look at the Microsoft examples, you will see that they declare the callbacks as WINAPI (without `extern "C"`): https://learn.microsoft.com/en-us/windows/win32/procthread/c...

So I stand by my point that in practice you don't need `extern "C"` for passing C++ callbacks to C functions. You can pass a lambda function just fine, and when it doesn't work the compiler will tell you.

Re: Rust for Filesystems

#194

Post by Firmwarrior »

abenga wrote: post #86
Firmwarrior wrote: post #69

Earlier quoted context omitted.

[flagged]

Hmmm. Is this serious or facetious? Or could be either, depending on the response? There already is rust code in the Linux kernel (some drivers), afaik.

Someone's gotta get the word out, I'm seriously sick of all these ridiculous LARPers trying to magic Rust into existence

Rust is a joke, it's a complete waste of time and effort to try and use it. It's the EU cookie law of native code - well-intentioned nonsense that doesn't work at all in reality

Re: Rust for Filesystems

#195

Post by LegibleCrimson2 »

spacechild1 wrote: post #193
LegibleCrimson2 wrote: post #187

Earlier quoted context omitted.

> `extern "C"` has nothing to do with linkage, all it does is disable namemangling, so you get the same symbol name as with a C compiler. extern "C" also ensures that the C calling convention is used, which is relevant for callbacks. It's not just name mangling. This is the reason that extern "C" static functions exist. You can actually overload a C++ function by extern "C" vs extern "C++", and it will dispatch it ap…

> extern "C" also ensures that the C calling convention is used, which is relevant for callbacks. It's not just name mangling. I stand corrected. I didn't know that `extern "C"` enforces the C calling convention. However, on modern platforms this doesn't really matter because, as I said, there is only a single calling convention (per platform). And I'm pretty sure that future platforms will keep it that way. Fortunat…

A couple big caveats here:

* cdecl is a platform specific calling convention. There is no standard C ABI. cdecl is a wintel thing, not the standard C calling convention. On Linux, this is the System V ABI for instance. On Windows ARM, it's also not cdecl.

* Specifying calling convention at all is a compiler specific extension. There is no standard way of specifying a C calling convention without `extern`.

So specifying cdecl gets you the right calling convention on some platforms and ties your code to some specific compilers. The only portable way to specify C linkage in a C++ program is extern "C". You will always get the right ABI for your platform and it will work on every compiler.

> So I stand by my point that in practice you don't need `extern "C"` for passing C++ callbacks to C functions. You can pass a lambda function just fine, and when it doesn't work the compiler will tell you.

The compiler will very often not tell you. It will complain if the lambda can't be coerced to a function pointer (because it's a closure) or if the argument or return types are wrong. An incorrect ABI will usually be accepted and will just do the wrong thing or crash at runtime. The C++ standard says that language linkage is part of a function's type, but very few compilers actually support this.

Your position works sometimes for some compilers and some platforms. I assert that it's better to use standard C++ features and just work everywhere.

Re: Rust for Filesystems

#196

Post by spacechild1 »

LegibleCrimson2 wrote: post #195
spacechild1 wrote: post #193

Earlier quoted context omitted.

> extern "C" also ensures that the C calling convention is used, which is relevant for callbacks. It's not just name mangling. I stand corrected. I didn't know that `extern "C"` enforces the C calling convention. However, on modern platforms this doesn't really matter because, as I said, there is only a single calling convention (per platform). And I'm pretty sure that future platforms will keep it that way. Fortunat…

A couple big caveats here: * cdecl is a platform specific calling convention. There is no standard C ABI. cdecl is a wintel thing, not the standard C calling convention. On Linux, this is the System V ABI for instance. On Windows ARM, it's also not cdecl. * Specifying calling convention at all is a compiler specific extension. There is no standard way of specifying a C calling convention without `extern`. So specifyi…

> * Specifying calling convention at all is a compiler specific extension.

Yes, because the calling conventions themselves are platform/compiler specific.

> There is no standard way of specifying a C calling convention without `extern`.

Well, on modern platforms you don't need to because there is only a single calling convention that is shared between C and C++. For legacy platforms with multiple calling conventions, you need compiler specific extensions by definition.

> The only portable way to specify C linkage in a C++ program is extern "C". You will always get the right ABI for your platform and it will work on every compiler.

Again, on platforms with several calling conventions `extern "C"` absolutely won't give you the appropriate calling convention all the time. See again my Win32 API example.

> The compiler will very often not tell you > An incorrect ABI will usually be accepted and will just do the wrong thing or crash at runtime.

That's absolutely not my experience! Functions with different calling conventions have different types, so a C++ compiler must reject such code. See https://godbolt.org/z/6EnncE5v5. (Note that for the lambda case MSVC is smart enough to automatically add __stdcall whereas MinGW refuses to compile. The free function is rejected by both compilers.)

Can you show me an actual example where a C++ compiler silently accepts a function with the wrong calling convention?

> Your position works sometimes for some compilers and some platforms.

It has always worked for me so far and I write software for many different platforms.

Re: Rust for Filesystems

#197

Post by LegibleCrimson2 »

spacechild1 wrote: post #196
LegibleCrimson2 wrote: post #195

Earlier quoted context omitted.

A couple big caveats here: * cdecl is a platform specific calling convention. There is no standard C ABI. cdecl is a wintel thing, not the standard C calling convention. On Linux, this is the System V ABI for instance. On Windows ARM, it's also not cdecl. * Specifying calling convention at all is a compiler specific extension. There is no standard way of specifying a C calling convention without `extern`. So specifyi…

> * Specifying calling convention at all is a compiler specific extension. Yes, because the calling conventions themselves are platform/compiler specific. > There is no standard way of specifying a C calling convention without `extern`. Well, on modern platforms you don't need to because there is only a single calling convention that is shared between C and C++. For legacy platforms with multiple calling conventions,…

Ah, yeah, you're right. I was spacing the fact that C as well as C++ can have multiple calling convention. I blame early morning brain.

As far as the wrong calling convention goes, I'm basing it on the fact that an extern "C++" function can be passed as a callback where an extern "C" is demanded. Even if they're the same calling convention, that should fail, but it doesn't. Looks like it doesn't fail at runtime, which is a small comfort, but given the different permissiveness of different compilers, it still makes me very nervous to pass a C++ function as a C callback and just hope that it works, given that it isn't guaranteed in the standard.

Re: Rust for Filesystems

#198

Post by spacechild1 »

LegibleCrimson2 wrote: post #197
spacechild1 wrote: post #196

Earlier quoted context omitted.

> * Specifying calling convention at all is a compiler specific extension. Yes, because the calling conventions themselves are platform/compiler specific. > There is no standard way of specifying a C calling convention without `extern`. Well, on modern platforms you don't need to because there is only a single calling convention that is shared between C and C++. For legacy platforms with multiple calling conventions,…

Ah, yeah, you're right. I was spacing the fact that C as well as C++ can have multiple calling convention. I blame early morning brain. As far as the wrong calling convention goes, I'm basing it on the fact that an extern "C++" function can be passed as a callback where an extern "C" is demanded. Even if they're the same calling convention, that should fail, but it doesn't. Looks like it doesn't fail at runtime, whic…

> Even if they're the same calling convention, that should fail, but it doesn't.

It's an interesting question. According to the standard, functions with different language linkage are indeed considered different types. As a consequence, should declare two overloads for qsort() that only differ in the type of the sort function. However, modern compilers don't seem to care:

"The only modern compiler that differentiates function types with "C" and "C++" language linkages is Oracle Studio, others do not permit overloads that are only different in language linkage, including the overload sets required by the C++ standard"

https://en.cppreference.com/w/cpp/language/language_linkage

In practice, extern "C" does two things (as you correctly pointed out):

1. disable name mangling - This only affects the symbol name and is not relevant for callback functions

2. enforce the (default) C calling convention - On all (modern) platforms I know, C and C++ have the same default calling convention for free functions.

This means that from the view of a C++ compiler, pointers to `foo()` and `extern "C" foo()` have the exact same type.

Anyway, no need to be nervous. Even if the compiler treated these as different types, you would get a compiler error because C++ disallows implicit casts between different pointer types.

Re: Rust for Filesystems

#199

Post by LegibleCrimson2 »

spacechild1 wrote: post #198
LegibleCrimson2 wrote: post #197

Earlier quoted context omitted.

Ah, yeah, you're right. I was spacing the fact that C as well as C++ can have multiple calling convention. I blame early morning brain. As far as the wrong calling convention goes, I'm basing it on the fact that an extern "C++" function can be passed as a callback where an extern "C" is demanded. Even if they're the same calling convention, that should fail, but it doesn't. Looks like it doesn't fail at runtime, whic…

> Even if they're the same calling convention, that should fail, but it doesn't. It's an interesting question. According to the standard, functions with different language linkage are indeed considered different types. As a consequence, should declare two overloads for qsort() that only differ in the type of the sort function. However, modern compilers don't seem to care: "The only modern compiler that differentiates…

As long as I can't silently get wrong behavior or runtime crashes, I'm happy enough. Is it guaranteed that an incorrect calling convention will always cause a compiler error? I wasn't aware the calling convention was considered part of the pointer type.

Anyway, thanks for engaging with me so earnestly. I guess I had some assumptions about calling conventions that needed to be straightened out, which is important, as I'm doing work in this territory right now.

Re: Rust for Filesystems

#200

Post by kweefee »

nicce wrote: post #154
mnau wrote: post #5

Earlier quoted context omitted.

> additional complexity tax Yes, but that should be offsetted by easier driver development. See the blog about Rust GPU driver for asahii linux, done in one month. EDIT: Google "tales of the m1 gpu" (author has a very negative opinions about hacker news, read if you like by clicking the link https://asahilinux.org/2022/11/tales-of-the-m1-gpu/ ) Is it universal? We'll see in coming years.

> author has a very negative opinions about hacker news I am not sure if author (Asahi Lina) has, but the project lead Hector Martin definitely has.

Same person.
Post reply on HN