Hobby x86 kernel written with Zig • Hacker News Board

Re: Hobby x86 kernel written with Zig

#121

Post by gw » Mon, Jan 06, 2020, 5:41 PM UTC

pron wrote: post #114

gw wrote: post #112

Earlier quoted context omitted.

English is certainly not free of ambiguity, but in the past i've seen you heavily emphasize precision in word choice, so it's surprising to see you de-emphasize it here. Nobody is a final arbiter of definitions, but the distinction i'm making is not a trivial one, and Hickey isn't the only one to have made it. Even thinking about it colloquially, how often do you follow the word "complex" with an infinitive verb desc…

I'm not deemphasizing it, I'm just saying that we're talking about different meanings of "complexity" here and there is no well-accepted definition. My "complexity" refers to the effort required by the programmer when understanding programs written in the language.

Fair enough, ron. I won't belabor it further. I'll just leave this: Long ago, after coming across a very useful distinction between the words "practical" and "pragmatic," i intentionally changed my usage of those words as a result. Not because a charismatic person told me to, but because it was useful. If a distinction is useful to make, start making it, my man!

Re: Hobby x86 kernel written with Zig

#122

Post by pron » Mon, Jan 06, 2020, 5:50 PM UTC

pcwalton wrote: post #120

pron wrote: post #113

Earlier quoted context omitted.

> The point is that Zig doesn't even try to prevent UAF now I wouldn't say Zig exists at all right now, but just as it strives to one day be production-ready, it strives to prevent use-after-free. Safety is a stated goal for the language. > Hasn't this essentially been what C++ has been trying for memory safety for decades, without success? No. I'm talking about a mechanism that can detect various errors at runtime,…

> No. I'm talking about a mechanism that can detect various errors at runtime, and is turned on or off for various pieces of code and/or for all code at various stages of development What you are describing exists: ASan. We have a pretty good answer to the question "is ASan sufficient to prevent memory safety problems in practice": "no, not really". > Rust, BTW, doesn't entirely guarantee memory-safety, either, when…

> We have a pretty good answer to the question "is ASan sufficient to prevent memory safety problems in practice"

That is not the question we're interested in answering, and elimination of all memory errors is no one's ultimate goal, certainly not at any cost. By definition, unsound techniques will let some errors through. The question is which approach leads to an overall safer program for a given effort, and soundness (of properties of interest) always comes at a cost.

(also, Zig catches various overflow errors better than ASan)

> Empirically, Rust's approach has resulted in far fewer memory safety problems than previous approaches like smart pointers

I don't doubt that, and if minimization of memory errors was programmers' primary concern (even in the scope of program correctness or even just security), there would be little doubt that Rust's approach is better.

As someone who currently mostly programs in C++, lack of memory safety barely makes my top three concerns. My #1 problem with C++ is that the language is far too complex for (my) comfort, where by "complex" I mean requires too much effort to read and write. That, and build times, have a bigger impact on the correctness of the programs I write than the lack of sound memory safety. Would I be happier if, for a similar cost, I could eliminate all memory safety errors? Sure, which is why, if C++ and Rust were the only low-level languages in existence, I'd rather people used Rust. But I would be happier still if I could solve the first two, and also get some better safety as a cherry-on-top. Memory safety is similarly not the main, and certainly not the only, reason I use languages that have a (tracing) GC when I use them.

Re: Hobby x86 kernel written with Zig

#123

Post by AndyKelley » Mon, Jan 06, 2020, 5:58 PM UTC

pcwalton wrote: post #100

simias wrote: post #79

Earlier quoted context omitted.

It's true that for bare metal programming Rust is quite far from 1.0. You can't even do inline assembly in stable rust... Arbitrary sized integers do sound very convenient, although I guess the counterargument would be that they simply can't map to machine types so you'll have to have magic behind the scenes which may not make complete sense in low level languages. Still, C has bitfields in structs which are sort of…

The problem with C-style bitfield values/arbitrary-sized integers is that they don't have addresses, so you can't take references to them. This means that you need special cases everywhere, and it increases the complexity of the language a lot. In Rust, integers often have methods that take self by reference, which demands an actual address. For these reasons, in Rust I think it's better that arbitrarily-sized intege…

In Zig they do; here is an example: https://clbin.com/HlJ0X

Output:

    Test [1/1] test "bit field access"...
    *align(:3:1) u3
    All 1 tests passed.

Zig's pointer type has optional metadata to describe alignment, sub-byte offset, and SIMD vector element index. This generally makes things just work, and you'll get a compile error if you expected an unadorned aligned pointer. If I were to swap:

    - fn deref(ptr: var) u3 {
    + fn deref(ptr: *u3) u3 {

Then I'd get:

    test.zig:13:23: error: expected type '*u3', found '*align(:3:1) u3'
        assert(deref(&data.b) == 2);
                          ^

Re: Hobby x86 kernel written with Zig

#124

Post by UncleOxidant » Mon, Jan 06, 2020, 6:02 PM UTC

jackhalford wrote: post #28

guggle wrote: post #6

Earlier quoted context omitted.

> I used to do this in Rust but I switched to zig for maintainability and readability over Rust. Can you expand on this ? I'm asking out of curiosity because I want to learn a "system" programming language (for whatever definition there is to this term). So far I briefly tried Rust and Nim and found the former more difficult to read. I know nothing about Zig, how would you place it between these two ?

In general you'll find that zig is easier to read than Rust (see the first version of this project in Rust [0]) because it's a simpler language. For kernel programming this is even more so the case: * zig has native support for arbitrary sized integers. In Rust I used to do bitshifts, Now I just have a packed struct of u3/u5/u7 whatever (see `src/pci/pci.zig`). Of course Rust has a bitflags crate but I didn't find it…

> * zig has native support for arbitrary sized integers

That's really cool.

Re: Hobby x86 kernel written with Zig

#125

Post by andi999 » Mon, Jan 06, 2020, 6:09 PM UTC

hippyhippo wrote: post #106

jackhalford wrote: post #81

Earlier quoted context omitted.

I don't follow JAI but maybe Jonathan Blow should try zig, it's a great fit for game development and I remember him mentioning that he wants a language that is fun to write in.

zig is following pretty much different path from jai afaik. zig prefers everything explicit and is very verbose whereas jai has many implicit things and also has macros or something similar. Anyways, more will be seen when it is released.

Maybe you can enlighten me, but from my view it seems the biggest difference is that zig exists and jai doesnt (yet?); at least for practical purposes.

Re: Hobby x86 kernel written with Zig

#126

Post by gameswithgo » Mon, Jan 06, 2020, 6:09 PM UTC

jackhalford wrote: post #81

rehemiau wrote: post #5

Earlier quoted context omitted.

Looks awesome! Planning to do a similar thing in Jai once it comes out

I don't follow JAI but maybe Jonathan Blow should try zig, it's a great fit for game development and I remember him mentioning that he wants a language that is fun to write in.

Blow is pretty familiar with Zig, there has been some collaboration between the two creators.

Re: Hobby x86 kernel written with Zig

#127

Post by vkaku » Mon, Jan 06, 2020, 6:10 PM UTC

jackhalford wrote: post #4

Hi, author here! I've just finished writing the pre-emptive multitasking [0](only round robbin though, nothing fancy). I'm currently writing an ATA driver [1], the idea is to implement ext2. I used to do this in Rust but I switched to zig for maintainability and readability over Rust. It seems that with `comptime` I'm able to make a lot of things optimal. Overall I have to say kernel programming is _hard_ but very re…

I'd rather you expose a FUSE interface first than expose ext2.... Just a totally random suggestion.

Re: Hobby x86 kernel written with Zig

#128

Post by AndyKelley » Mon, Jan 06, 2020, 6:18 PM UTC

blackhaz wrote: post #20

nicoburns wrote: post #10

Earlier quoted context omitted.

Zig is lower level than either (it's more like C than C++: expect to manually call 'free'on allocated values). It has a lot of nice improvments over C however, like proper arrays, no null, a module system, and proper compile time evaluation (no need for the hacky preprocessor).

May I ask why people invent new languages rather than trying to extend or improve existing ones, like C?

Here is a talk I gave that directly addresses this question:

https://www.youtube.com/watch?v=Gv2I7qTux7g

(the title & abstract in the youtube description is the one I gave for the RFP; I ended up going in a slightly different direction than it once I actually made the talk)

Re: Hobby x86 kernel written with Zig

#129

Post by DSMan195276 » Mon, Jan 06, 2020, 6:23 PM UTC

rapsey wrote: post #58

woodrowbarlow wrote: post #56

Earlier quoted context omitted.

yes, that's the point i'm making. saying "if you write your kernel in rust, it will be memory safe" is a little goofy, because writing kernels in rust requires you to drop into unsafe blocks way too often.

The ammount of unsafe blocks is pretty small compared to the rest of the code. Which means those parts can be reviewed more extensively. If anything your argument is pretty goofy.

> The ammount of unsafe blocks is pretty small compared to the rest of the code. Which means those parts can be reviewed more extensively. If anything your argument is pretty goofy.

I've talked about this at length before, but just because you have 'less' `unsafe` code does not mean you code is any better off. Very few operations are actually `unsafe`, and you're still not allowed to break any of the (poorly or not documented) constraints that Rust requires even in `unsafe` code.

Point being, unless you can make "safe" abstractions around your `unsafe` code that cannot be broken (Which is largely what you do in any language anyway, when you can...), the distinction between "safe" and "unsafe" is thin, because the parts of the code you declare `unsafe` are not the parts that are actually going to have the bugs. And because of the fact that the constraints you are required to fulfill in `unsafe` code is unclear, there's little way to guarantee your code is broken in some subtle way the optimizer might jump on (either now, or in a later release).

And to be clear, I like Rust, but `unsafe` is poorly thought-out IMO.

Re: Hobby x86 kernel written with Zig

#130

Post by apta » Mon, Jan 06, 2020, 6:32 PM UTC

pron wrote: post #122

pcwalton wrote: post #120

Earlier quoted context omitted.

> No. I'm talking about a mechanism that can detect various errors at runtime, and is turned on or off for various pieces of code and/or for all code at various stages of development What you are describing exists: ASan. We have a pretty good answer to the question "is ASan sufficient to prevent memory safety problems in practice": "no, not really". > Rust, BTW, doesn't entirely guarantee memory-safety, either, when…

> We have a pretty good answer to the question "is ASan sufficient to prevent memory safety problems in practice" That is not the question we're interested in answering, and elimination of all memory errors is no one's ultimate goal, certainly not at any cost. By definition, unsound techniques will let some errors through. The question is which approach leads to an overall safer program for a given effort, and soundn…

> As someone who currently mostly programs in C++, lack of memory safety barely makes my top three concerns.

Isn't having a UB-free JVM a noteworthy goal though? Especially if it gets used in life-critical systems such as avionics or autonomous cars.