Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's New Async I/O

kristoff.it

Post reply on HN
Print view View original

161–170 of 293 posts

Re: Zig's New Async I/O

#161

Post by mlugg »

logicchains wrote: post #159

Does this mean that as a side effect, it'll now be possible to enforce functions are pure/deterministic in Zig by not passing in an Io?

Not quite:

* Global variables still exist and can be stored to / loaded from by any code

* Only convention stops a function from constructing its own `Io`

* Only convention stops a function from reaching directly into low-level primitives (e.g. syscalls or libc FFI)

However, in practice, we've found that such conventions tend to be fairly well-respected in most Zig code. I anticipate `Io` being no different. So, if you see a function which doesn't take `Io`, you can be pretty confident (particularly if it's in a somewhat reputable codebase!) that it's not interacting with the system (e.g. doing filesystem accesses, opening sockets, sleeping the thread).

Re: Zig's New Async I/O

#162

Post by masklinn »

dwattttt wrote: post #125
masklinn wrote: post #82

Earlier quoted context omitted.

It’s more like adding a runtime handle to the struct. Modulo that I’m not sure any langage with a sync/async split has an “async” runtime built entirely out of sync operations. So a library can’t take a runtime for a caller and get whatever implementation the caller decided to use.

> I’m not sure any langage with a sync/async split has an “async” runtime built entirely out of sync operations. You get into hairy problems of definition, but you can definitely create an "async" runtime out of "sync" operations: implement an async runtime with calls to C. C doesn't have a concept of "async", and more or less all async runtime end up like this. I've implemented Future (Rust) on a struct for a Window…

> You get into hairy problems of definition, but you can definitely create an "async" runtime out of "sync" operations: implement an async runtime with calls to C. C doesn't have a concept of "async", and more or less all async runtime end up like this.

While C doesn't have async OS generally provide APIs which are non-blocking, and that is what async runtimes are implemented on top of.

By sync operations I mean implementing an "async" runtime entirely atop blocking operations, without bouncing them through any sort of worker threads or anything.

Re: Zig's New Async I/O

#163

Post by dminik »

mlugg wrote: post #154
dminik wrote: post #144

Earlier quoted context omitted.

It's funny, but I do actually like it. It's just that it walks like a duck, swims like a duck and quacks like a duck. I don't have a problem with IO conceptually (but I do have a problem with Zig ergonomics, allocator included). I do have a problem with claiming you defeated function coloring. Like, look. You didn't even get rid of await ... > try a_future.await(io);

I mean... you use `await` if you've used `async`. It's your choice whether or not you do; and if you don't want to, your callers and callees can still freely `async` and `await` if they want to. I don't understand the point you're trying to make here. To be clear, where many languages require you to write `const x = await foo()` every time you want to call an async function, in Zig that's just `const x = foo()`. This…

> in Zig that's just ...

Well, no. In zig that's `const x = foo(io)`.

The moment you take or even know about an io, your function is automatically "generic" over the IO interface.

Using stackless coroutines and green threads results in a completely different codegen.

I just noticed this part of the article:

> Stackless Coroutines > > This implementation won’t be available immediately like the previous ones because it depends on reintroducing a special function calling convention and rewriting function bodies into state machines that don’t require an explicit stack to run. > > This execution model is compatible with WASM and other platforms where stack swapping is not available or desireable.

I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.

Re: Zig's New Async I/O

#164

Post by sapiogram »

schrodinger wrote: post #44
tidwall wrote: post #36

Earlier quoted context omitted.

Context is not required in Go and I personally encourage you to avoid it. There is no shame in blazing a different path.

Why do you encourage avoiding it? Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm. To be very clear, I'm asking this completely in good faith looking to learn something new!

> Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm.

I'm not sure what you mean here. Preemptive/coorporative terminology refers to interrupting (not aborting) a CPU-bound task, in which case goroutines are fully preemptive on most platforms since Go 1.14, check the release notes for more info. However, this has nothing to do with context.

If you're referring to early-aborting IO operations, then yes, that's what context is for. However, this doesn't really have anything to do with goroutines, you could do the same if the runtime was built on OS threads.

Re: Zig's New Async I/O

#165

Post by SkiFire13 »

mlugg wrote: post #8
dminik wrote: post #3

I feel that I have to point this out once again, because the article goes so far as to state that: > With this last improvement Zig has completely defeated function coloring. I disagree with this. Let's look at the 5 rules referenced in the famous "What color is your function?" article referenced here. > 1. Every function has a color Well, you don't have async/sync/red/blue anymore, but you now have IO and non-IO fun…

The key difference to typical async function coloring is that `Io` isn't something you need specifically for asynchronicity; it's something which (unless you make a point to reach into very low-level primitives) you will need in order to perform any IO, including reading a file, sleeping, getting the time, etc. It's also just a value which you can keep wherever you want, rather than a special attribute/property of a…

> I do agree that in principle, there's still a form of function coloring going on. Arguably, our solution to the problem is just to color every function async-colored

I feel like there are two issues with this approach:

- you basically rely on the compiler/stdlib to silently switch the async implementation, effectively implementing a sort of hidden control flow which IMO doesn't really fit Zig

- this only solves the "visible" coloring issue of async vs non-async functions, but does not try to handle the issue of blocking vs non-blocking functions, rather it hides it by making all functions have the same color

- you're limiting the set of async operations to the ones supported in the `Io`'s vtable. This forces it to e.g. include mutexes, even though they are not really I/O, because they might block and hence need async support. But if I wrote my own channel how would this design support it?

Re: Zig's New Async I/O

#166

Post by mlugg »

dminik wrote: post #163
mlugg wrote: post #154

Earlier quoted context omitted.

I mean... you use `await` if you've used `async`. It's your choice whether or not you do; and if you don't want to, your callers and callees can still freely `async` and `await` if they want to. I don't understand the point you're trying to make here. To be clear, where many languages require you to write `const x = await foo()` every time you want to call an async function, in Zig that's just `const x = foo()`. This…

> in Zig that's just ... Well, no. In zig that's `const x = foo(io)`. The moment you take or even know about an io, your function is automatically "generic" over the IO interface. Using stackless coroutines and green threads results in a completely different codegen. I just noticed this part of the article: > Stackless Coroutines > > This implementation won’t be available immediately like the previous ones because it…

> Well, no. In zig that's `const x = foo(io)`.

If `foo` needs to do IO, sure. Or, more typically (as I mentioned in a different comment), it's something like `const x = something.foo()`, and `foo` can get its `Io` instance from `something` (in the Zig compiler this would be a `Compilation` or a `Zcu` or a `Sema` or something like that).

> Using stackless coroutines and green threads results in a completely different codegen.

Sure, but that's abstracted away from you. To be clear, stackless coroutines are the only case where the codegen of callers is affected, which is why they require a language feature. Even if your application uses two `Io` implementations for some reason, one of which is based on stackless coroutines, functions using the API are not duplicated.

> I wonder what will happen if you try to await a future created with a green thread IO using a stackless coroutine IO.

Mixing futures from any two different `Io` implementations will typically result in Illegal Behavior -- just like passing a pointer allocated with one `Allocator` into the `free` of a different `Allocator` does. This really isn't a problem. Even with allocators, it's pretty rare for people to mess this up, and with allocators you often do have multiple of them available in one place (e.g. a gpa and an arena). In contrast, it will be extraordinarily rare to have more than one `Io` lying around. Even if you do mess it up, the IB will probably just trip a safety check, so it shouldn't take you too long to realise what you've done.

Re: Zig's New Async I/O

#167

Post by pimeys »

kristoff_it wrote: post #112
com2kid wrote: post #81

Earlier quoted context omitted.

> Isn't "coloring" just another form of static typing? Yes, and so is declaring what exceptions a function can throw (checked exceptions in Java). > Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different? It really isn't a problem. The article makes people th…

> but IMHO people who sit down for a bit and think through the issue come to the same conclusion you have - Function coloring isn't a problem in practice. I dunno man, have you seen people complain about async virality in Rust being annoying? Have you ever tried to read a backtrace from a program that does stackless coroutines (it's not fun)? Have you seen people do basically duplicate work to maintain a blocking and…

People do complain, like they do from things like systemd. Then there is us, the silent majority, who just get shit done with these tools.

I respect what zig has done here, and I will want to try it out when it stabilizes. But Rust async is just fine.

Re: Zig's New Async I/O

#168

Post by krior »

ksec wrote: post #68
mlugg wrote: post #11

Earlier quoted context omitted.

I'm not sure how you got the perception that we're going "all in" on green threads, given that the article in OP explicitly mentions that we're hoping to have an implementation based on stackless coroutines, based on this Zig language proposal: https://github.com/ziglang/zig/issues/23446 Performance matters; we're not planning to forget that. If fibers turn out to have unacceptable performance characteristics, then t…

That is lovely to hear. I think the general conscious is that not a single programming language has done Async right. So people are a little sceptical. But Andrew and the team so far seems to have the do it right mentality. So I guess people should be a little more optimistic. Cant wait for 0.15 coming out soon.

I would argue that Java's async is pretty nifty, especially now that some of the rough edges have been sanded off.

Re: Zig's New Async I/O

#169

Post by [deleted] »

chrisohara wrote: post #151
laserbeam wrote: post #103

Earlier quoted context omitted.

> If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too. Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things. Also, it’s perfectly reasonable and sometimes desireable to have 13 different allocators in your program at once. Short lived ones, long lived ones, temporary al…

> Io in zig is for “things that can block execution”. Things that could semantically cause a yield of any kind. Allocation is not one of those things. The allocator may yield to the OS when requesting or releasing memory (e.g. sbrk, mmap, munmap)?

[deleted]

Re: Zig's New Async I/O

#170

Post by throwawaymaths »

camgunz wrote: post #79
throwawaymaths wrote: post #67

Earlier quoted context omitted.

> But I guess it's going to be 2 params from now on. >> So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io

Python, for example, will let you call async functions inside non-async functions, you just have to set up the event loop yourself. This isn't conceptually different than the Io thing here.

except you cant "pass the same event loop in multiple locations". its also not an easy lift. the zig std will provide a few standard implementations which would be trivial to drop in.
Post reply on HN