Hobby x86 kernel written with Zig • Hacker News Board

Re: Hobby x86 kernel written with Zig

#111

Post by pcwalton » Mon, Jan 06, 2020, 4:58 PM UTC

pron wrote: post #110

pcwalton wrote: post #102

Earlier quoted context omitted.

> There is also the difference in their approach to safety, but that's a complicated subject that ultimately boils down to an empirical question -- which approach is safer? -- which we don't have the requisite data to answer. We do have the requisite data to answer whether preventing use-after-free is better than not preventing it. You can argue (not successfully, in my opinion) that it's not worth the loss in produc…

> We do have the requisite data to answer whether preventing use-after-free is better than not preventing it. I expect Zig will prevent use-after-free. It will be sound for safe code and unsound for unsafe code (by turning this on only in debug mode for testing). > but it's impossible to argue that not trying to prevent UAF is somehow safer. First, see above. Second, it is not only possible but even reasonable to arg…

It's not interesting to talk about a system that might or might not exist in the future. (There is a word for that—"vaporware".) The point is that Zig doesn't even try to prevent UAF now, so you can't say that it's safer than languages that do prevent the problem.

> As an example, instead of soundly eliminating bugs of kind A, reducing bugs of kinds A, B and C -- for a similar cost -- may well be safer.

Hasn't this essentially been what C++ has been trying for memory safety for decades, without success? The C++ approach has been "smart pointers are good enough, and they prevent several other problems too", and the experience of web browsers (among others) has pretty much definitively shown: no, they really aren't. For memory safety, I would not bet on this approach.

Re: Hobby x86 kernel written with Zig

#112

Post by gw » Mon, Jan 06, 2020, 5:00 PM UTC

pron wrote: post #108

gw wrote: post #107

Earlier quoted context omitted.

Then you aren't talking about complexity (an objective quality), you are talking about difficulty to read (a subjective quality relative to the reader). There is no doubt that macros can make a given piece of code harder to read, if the reader is unfamiliar with the macro being used. Complexity describes how intertwined different pieces of something are internally, which has nothing to do with a given vantage point.

No, that's just how Rich Hickey describes complexity; it's hardly a universal definition. For example, in computer science, the complexity of a task is often a measure of the effort, in time or memory, required to perform it.

English is certainly not free of ambiguity, but in the past i've seen you heavily emphasize precision in word choice, so it's surprising to see you de-emphasize it here. Nobody is a final arbiter of definitions, but the distinction i'm making is not a trivial one, and Hickey isn't the only one to have made it. Even thinking about it colloquially, how often do you follow the word "complex" with an infinitive verb describing an action? A rube goldberg machine is complex...and hard to build!

Re: Hobby x86 kernel written with Zig

#113

Post by pron » Mon, Jan 06, 2020, 5:01 PM UTC

pcwalton wrote: post #111

pron wrote: post #110

Earlier quoted context omitted.

> We do have the requisite data to answer whether preventing use-after-free is better than not preventing it. I expect Zig will prevent use-after-free. It will be sound for safe code and unsound for unsafe code (by turning this on only in debug mode for testing). > but it's impossible to argue that not trying to prevent UAF is somehow safer. First, see above. Second, it is not only possible but even reasonable to arg…

It's not interesting to talk about a system that might or might not exist in the future. (There is a word for that—"vaporware".) The point is that Zig doesn't even try to prevent UAF now, so you can't say that it's safer than languages that do prevent the problem. > As an example, instead of soundly eliminating bugs of kind A, reducing bugs of kinds A, B and C -- for a similar cost -- may well be safer. Hasn't this e…

> The point is that Zig doesn't even try to prevent UAF now

I wouldn't say Zig exists at all right now, but just as it strives to one day be production-ready, it strives to prevent use-after-free. Safety is a stated goal for the language.

> Hasn't this essentially been what C++ has been trying for memory safety for decades, without success?

No. I'm talking about a mechanism that can detect various errors at runtime, and is turned on or off for various pieces of code and/or for all code at various stages of development. Rust, BTW, doesn't entirely guarantee memory-safety, either, when any unproven unsafe code is used, and even when it isn't (e.g., have you proven LLVM's correctness?). We always make some compromises on soundness; the question is where the sweet-spots are.

Software correctness is one area where there are no easy answers and very few obvious ones.

Re: Hobby x86 kernel written with Zig

#114

Post by pron » Mon, Jan 06, 2020, 5:09 PM UTC

gw wrote: post #112

pron wrote: post #108

Earlier quoted context omitted.

No, that's just how Rich Hickey describes complexity; it's hardly a universal definition. For example, in computer science, the complexity of a task is often a measure of the effort, in time or memory, required to perform it.

English is certainly not free of ambiguity, but in the past i've seen you heavily emphasize precision in word choice, so it's surprising to see you de-emphasize it here. Nobody is a final arbiter of definitions, but the distinction i'm making is not a trivial one, and Hickey isn't the only one to have made it. Even thinking about it colloquially, how often do you follow the word "complex" with an infinitive verb desc…

I'm not deemphasizing it, I'm just saying that we're talking about different meanings of "complexity" here and there is no well-accepted definition. My "complexity" refers to the effort required by the programmer when understanding programs written in the language.

Re: Hobby x86 kernel written with Zig

#115

Post by CDSlice » Mon, Jan 06, 2020, 5:10 PM UTC

jackhalford wrote: post #90

simias wrote: post #79

Earlier quoted context omitted.

It's true that for bare metal programming Rust is quite far from 1.0. You can't even do inline assembly in stable rust... Arbitrary sized integers do sound very convenient, although I guess the counterargument would be that they simply can't map to machine types so you'll have to have magic behind the scenes which may not make complete sense in low level languages. Still, C has bitfields in structs which are sort of…

Ok maybe I was user pointers wrong in Rust. And I must confess that I never really understood lifetimes either, however * I don't think that low level code needs to be "messy", and * I don't think that adding an abstraction on top solves anything complexity wise. For memory paging I used to have a whole library in Rust that handled hierarchy of page tables with very abstract pagetable classes and interfaces [0]. Now…

The Rust code you wrote seems to be extremely heavy on boilerplate. A couple of macros could have drastically reduced the line count.

I don't think what Linus wrote in 2004 on C++ has much relevance for 2020 Rust, especially for a new kernel that doesn't have a ton of contributors. Most of his complaints seem to be on how C++ abstractions can make it hard to review unknown code and how 2004 C++ code bases often had extremely messy code. Rust IMO doesn't suffer from those problems near as much as 2004 C++ did.

Also, Google is using Rust for part of their new Fuchsia kernel so at least they think Rust has something good to offer kernel dev.

Re: Hobby x86 kernel written with Zig

#116

Post by jeltz » Mon, Jan 06, 2020, 5:15 PM UTC

blackhaz wrote: post #20

nicoburns wrote: post #10

Earlier quoted context omitted.

Zig is lower level than either (it's more like C than C++: expect to manually call 'free'on allocated values). It has a lot of nice improvments over C however, like proper arrays, no null, a module system, and proper compile time evaluation (no need for the hacky preprocessor).

May I ask why people invent new languages rather than trying to extend or improve existing ones, like C?

Yeah, why did people not fix ALGOL instead of writing new ones like C?

Re: Hobby x86 kernel written with Zig

#117

Post by Touche » Mon, Jan 06, 2020, 5:17 PM UTC

pcwalton wrote: post #111

pron wrote: post #110

Earlier quoted context omitted.

> We do have the requisite data to answer whether preventing use-after-free is better than not preventing it. I expect Zig will prevent use-after-free. It will be sound for safe code and unsound for unsafe code (by turning this on only in debug mode for testing). > but it's impossible to argue that not trying to prevent UAF is somehow safer. First, see above. Second, it is not only possible but even reasonable to arg…

It's not interesting to talk about a system that might or might not exist in the future. (There is a word for that—"vaporware".) The point is that Zig doesn't even try to prevent UAF now, so you can't say that it's safer than languages that do prevent the problem. > As an example, instead of soundly eliminating bugs of kind A, reducing bugs of kinds A, B and C -- for a similar cost -- may well be safer. Hasn't this e…

Just for clarity for anyone reading, the Zig author does not claim that Zig is safe and has in fact said that is unsafe. Could change in the future, but there's no denial about what it is today.

Re: Hobby x86 kernel written with Zig

#118

Post by pron » Mon, Jan 06, 2020, 5:27 PM UTC

Touche wrote: post #117

pcwalton wrote: post #111

Earlier quoted context omitted.

It's not interesting to talk about a system that might or might not exist in the future. (There is a word for that—"vaporware".) The point is that Zig doesn't even try to prevent UAF now, so you can't say that it's safer than languages that do prevent the problem. > As an example, instead of soundly eliminating bugs of kind A, reducing bugs of kinds A, B and C -- for a similar cost -- may well be safer. Hasn't this e…

Just for clarity for anyone reading, the Zig author does not claim that Zig is safe and has in fact said that is unsafe. Could change in the future, but there's no denial about what it is today.

There is a difference between safe code, which is the goal, and a safe language (that's a statement the language makes on sound safety guarantees). Using a safe language is definitely one way to write safe code, but it is not necessarily always the best way, and it's certainly not the only way. Zig is not meant to ever be a safe language, but it is very much intended to be a language that helps write safe code. That is what I meant when I said that the two languages have a very different approach to safety.

Re: Hobby x86 kernel written with Zig

#119

Post by roblabla » Mon, Jan 06, 2020, 5:29 PM UTC

CDSlice wrote: post #115

jackhalford wrote: post #90

Earlier quoted context omitted.

Ok maybe I was user pointers wrong in Rust. And I must confess that I never really understood lifetimes either, however * I don't think that low level code needs to be "messy", and * I don't think that adding an abstraction on top solves anything complexity wise. For memory paging I used to have a whole library in Rust that handled hierarchy of page tables with very abstract pagetable classes and interfaces [0]. Now…

The Rust code you wrote seems to be extremely heavy on boilerplate. A couple of macros could have drastically reduced the line count. I don't think what Linus wrote in 2004 on C++ has much relevance for 2020 Rust, especially for a new kernel that doesn't have a ton of contributors. Most of his complaints seem to be on how C++ abstractions can make it hard to review unknown code and how 2004 C++ code bases often had e…

Small nitpick: Google isn't using Rust for the Fuchsia kernel. The kernel, Zircon, is written entirely in C and is based on Little Kernel. Google is using Rust for some userspace drivers (Fuchsia being a micro-kernel).

Re: Hobby x86 kernel written with Zig

#120

Post by pcwalton » Mon, Jan 06, 2020, 5:34 PM UTC

pron wrote: post #113

pcwalton wrote: post #111

Earlier quoted context omitted.

It's not interesting to talk about a system that might or might not exist in the future. (There is a word for that—"vaporware".) The point is that Zig doesn't even try to prevent UAF now, so you can't say that it's safer than languages that do prevent the problem. > As an example, instead of soundly eliminating bugs of kind A, reducing bugs of kinds A, B and C -- for a similar cost -- may well be safer. Hasn't this e…

> The point is that Zig doesn't even try to prevent UAF now I wouldn't say Zig exists at all right now, but just as it strives to one day be production-ready, it strives to prevent use-after-free. Safety is a stated goal for the language. > Hasn't this essentially been what C++ has been trying for memory safety for decades, without success? No. I'm talking about a mechanism that can detect various errors at runtime,…

> No. I'm talking about a mechanism that can detect various errors at runtime, and is turned on or off for various pieces of code and/or for all code at various stages of development

What you are describing exists: ASan. We have a pretty good answer to the question "is ASan sufficient to prevent memory safety problems in practice": "no, not really".

> Rust, BTW, doesn't entirely guarantee memory-safety, either, when any unproven unsafe code is used, and even when it isn't (e.g., have you proven LLVM's correctness?). We always make some compromises on soundness; the question is where the sweet-spots are.

Empirically, Rust's approach has resulted in far fewer memory safety problems than previous approaches like smart pointers and ASan, with only garbage collectors (and restrictive languages with no allocation at all) having similar success in practice. Notice that the working approaches have something important in common: a strong system that, given certain assumptions, guarantees the lack of memory safety problems. Even though those assumptions are never quite satisfied in practice, empirically having those theoretical guarantees seems important. It separates systems that drastically reduce safety problems, such as Rust and GC languages, from those that do so less well, such as ASan and smart pointers. This is why I'm so skeptical of just piling on more mitigations: they're helpful, but we've been piling on mitigations for decades and UAF (for instance) is still a big a problem as ever.