Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's New Async I/O

kristoff.it

Post reply on HN
Print view View original

171–180 of 293 posts

Re: Zig's New Async I/O

#171

Post by laserbeam »

chrisohara wrote: post #151
laserbeam wrote: post #103

Earlier quoted context omitted.

> If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too. Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things. Also, it’s perfectly reasonable and sometimes desireable to have 13 different allocators in your program at once. Short lived ones, long lived ones, temporary al…

> Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things. The allocator may yield to the OS when requesting or releasing memory (e.g. sbrk, mmap, munmap)?

Yielding in this context means to a different “thread” in your context, not the OS. If you want to express “this is a point where the program can do something else” it is a yield. If you block and can’t switch to something else… it is not.

So if you’re using an API like mmap like that you should think of it as IO (I don’t think you can, but am not sure).

Re: Zig's New Async I/O

#173

Post by throwawaymaths »

laserbeam wrote: post #171
chrisohara wrote: post #151

Earlier quoted context omitted.

> Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things. The allocator may yield to the OS when requesting or releasing memory (e.g. sbrk, mmap, munmap)?

Yielding in this context means to a different “thread” in your context, not the OS. If you want to express “this is a point where the program can do something else” it is a yield. If you block and can’t switch to something else… it is not. So if you’re using an API like mmap like that you should think of it as IO (I don’t think you can, but am not sure).

the page allocator which is the root of many allocators calls mmap. of course the fixed buffer allocator does not.

Re: Zig's New Async I/O

#174

Post by throwawaymaths »

n42 wrote: post #12
dminik wrote: post #3

I feel that I have to point this out once again, because the article goes so far as to state that: > With this last improvement Zig has completely defeated function coloring. I disagree with this. Let's look at the 5 rules referenced in the famous "What color is your function?" article referenced here. > 1. Every function has a color Well, you don't have async/sync/red/blue anymore, but you now have IO and non-IO fun…

Aside from the ridiculous argument that function parameters color them, the assertion that you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in

> you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in

that's not true. suppose a function foo(anytype) takes a struct, and expects method bar() on the struct.

you could send foo() the struct type Sync whose bar() does not use io. or you could send foo() the struct type Async whose bar uses an io stashed in the parameter, and there would be no code changes.

if you don't prefer compile time multireification, you can also use type erasure and accomplish the same thing with a vtable.

Re: Zig's New Async I/O

#175

Post by dminik »

mlugg wrote: post #166
dminik wrote: post #163

Earlier quoted context omitted.

> in Zig that's just ... Well, no. In zig that's `const x = foo(io)`. The moment you take or even know about an io, your function is automatically "generic" over the IO interface. Using stackless coroutines and green threads results in a completely different codegen. I just noticed this part of the article: > Stackless Coroutines > > This implementation won’t be available immediately like the previous ones because it…

> Well, no. In zig that's `const x = foo(io)`. If `foo` needs to do IO, sure. Or, more typically (as I mentioned in a different comment), it's something like `const x = something.foo()`, and `foo` can get its `Io` instance from `something` (in the Zig compiler this would be a `Compilation` or a `Zcu` or a `Sema` or something like that). > Using stackless coroutines and green threads results in a completely different…

I find these two statements to be contradictory

> Sure, but that's abstracted away from you

> Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior

Thinking about it more, you've possibly added even more colors. Each executor adds a different color and while each function is color-agnostic (but not colorless) futures aren't.

> it will be extraordinarily rare to have more than one `Io`

Will it? I can immediately think of a use case where a program might want to block for files on disk, but defer fetching from network to some background async executor.

Re: Zig's New Async I/O

#176

Post by kbolino »

sapiogram wrote: post #164
schrodinger wrote: post #44

Earlier quoted context omitted.

Why do you encourage avoiding it? Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm. To be very clear, I'm asking this completely in good faith looking to learn something new!

> Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm. I'm not sure what you mean here. Preemptive/coorporative terminology refers to interrupting (not aborting) a CPU-bound task, in which case goroutines are fully preemptive on most platforms since Go 1.14, check the release notes for more info. However, this has nothing to do with context. If you'r…

Goroutines are preemptive only to the runtime scheduler. You, the application developer merely using the language, cannot directly preempt a goroutine.

This makes goroutines effectively cooperative still from the perspective of the developer. The preemptive runtime "just" prevents things like user code starving out the garbage collector. To interrupt a goroutine, your options are generally limited to context cancelation and closing the channel or socket being read, if any. And the goroutine may still refuse to exit (or whatever else you want it to do), though that's largely up to how you code it.

This difference is especially stark when compared with Erlang/BEAM where you can directly address, signal, and terminate its lightweight processes.

Re: Zig's New Async I/O

#177

Post by Icathian »

mlugg wrote: post #150
comex wrote: post #77

Earlier quoted context omitted.

But how will that actually work? Your stackless coroutines proposal talks about explicit primitives for defining a coroutine. But what about a function that's not designed for any particular implementation strategy - it just takes an Io and passes it on to some other functions? Will the compiler have a way to compile it as either sync or async, like apparently it did before? It would have to, if you want to avoid fun…

Right, the proposal doesn't discuss the implementation details -- I do apologise if that made it seem a little hand-wavey. I opted not to discuss them there, because they're similar-ish to the way we lowered stackless async in its stage1 implementation, and hence not massively interesting to discuss. The idea is that, yes, the compiler will infer whether or not a function is async (in the stackless async sense) based…

> a change in a leaf function in a little file in a random helper module could introduce asynchronocity which propagates all the way up to your `pub fn main`

If this doesn't make the argument that Zig has certainly not defeated function coloring, I don't know what would.

The fact that this change to how my program runs is hidden away in the compiler instead of somewhere visible is not an improvement.

Re: Zig's New Async I/O

#179

Post by xmorse »

This is a good time to implement "context", a way to pass down the call stack parameters instead of having to add a io argument to every function
Post reply on HN