Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's new plan for asynchronous programs

lwn.net

Post reply on HN
Print view View original

131–140 of 274 posts

Re: Zig's new plan for asynchronous programs

#131

Post by torginus »

messe wrote: post #97
tcfhgj wrote: post #93

Earlier quoted context omitted.

what about non-io code?

What about it? It gets called without an Io parameter. Same way that a function that doesn't allocate doesn't get an allocator. I feel like you're trying to set me up for a gotcha "see, zig does color functions because it distinguishes functions that do io and those that don't!". And yes, that's true. Zig, at least Zig code using std, will mark functions that do Io with an Io parameter. But surely you can see how tha…

This creates the drill-down issue we see with React props where we have to pass objects around in the call chain just so that somewhere down the line we can use it.

React gets around this with the context hook and which you can access implicitly if it has been injected at a higher level.

Do you know if Zig supports something of the sort?

Re: Zig's new plan for asynchronous programs

#132

Post by woodruffw »

eikenberry wrote: post #122
woodruffw wrote: post #22

I think this design is very reasonable. However, I find Zig's explanation of it pretty confusing: they've taken pains to emphasize that it solves the function coloring problem, which it doesn't: it pushes I/O into an effect type, which essentially behaves as a token that callers need to retain. This is a form of coloring, albeit one that's much more ergonomic. (To my understanding this is pretty similar to how Go sol…

Function coloring is specifically about requiring syntax for a function, eg. the async keyword. So if you want an async and non-async function you need to write both in code. If you pass the "coloring" as an argument you avoid the need for extra syntax and multiple function definitions and therefor the function has no color. You can solve this in various ways with various tradeoffs but as long as there is a single fu…

> Function coloring is specifically about requiring syntax for a function, eg. the async keyword.

Someone should tell the inventor of the phrase, because they don't mention the async keyword at all[1]. As-written, function coloring is about callbacks (since that's semantic mechanism that JavaScript happens to pick for their asynchronous model).

Function coloring is just an informal way to describe encoding a function's effect. You can encode that in syntax if you want (an `async` keyword), or in the type system (returning `() -> T` instead of `T`), or in the runtime itself (by controlling all I/O and treating it the same). But you can't avoid it.

[1]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Re: Zig's new plan for asynchronous programs

#133

Post by throwawaymaths »

woodruffw wrote: post #22

I think this design is very reasonable. However, I find Zig's explanation of it pretty confusing: they've taken pains to emphasize that it solves the function coloring problem, which it doesn't: it pushes I/O into an effect type, which essentially behaves as a token that callers need to retain. This is a form of coloring, albeit one that's much more ergonomic. (To my understanding this is pretty similar to how Go sol…

1) zig's io is not a viral effect type, you can in principle declare a global io variable and use it everywhere that any library calls for it. Not best practice for a library writer, but if you're building an app, do what you want.

2) There are two things here, there is function coloring and the function coloring problem. The function coloring problem is five things:

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

1. Every function has a color.

2. The way you call a function depends on its color.

3. You can only call a red function from within another red function.

4. Red functions are more painful to call.

5. Some core library functions are red.

You'll have some convincing to do that zig's plan satisfies 4. It's almost certain that it won't satisfy 5.

It's open to debate if zig's plan will work at all, of course.

Re: Zig's new plan for asynchronous programs

#134

Post by throwawaymaths »

SkiFire13 wrote: post #84
flohofwoe wrote: post #34

Earlier quoted context omitted.

If calling the same function with a different argument would be considered 'function coloring', every function in a program is 'colored' and the word loses its meaning ;) Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a…

> Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit). AFAIK this still leaked through function pointers, which were still sync or async (and this was not visible in their type)

Pretty sure the Zig team is aware of this and has plans to fix it before they re-release async.

Re: Zig's new plan for asynchronous programs

#135

Post by messe »

torginus wrote: post #131
messe wrote: post #97

Earlier quoted context omitted.

What about it? It gets called without an Io parameter. Same way that a function that doesn't allocate doesn't get an allocator. I feel like you're trying to set me up for a gotcha "see, zig does color functions because it distinguishes functions that do io and those that don't!". And yes, that's true. Zig, at least Zig code using std, will mark functions that do Io with an Io parameter. But surely you can see how tha…

This creates the drill-down issue we see with React props where we have to pass objects around in the call chain just so that somewhere down the line we can use it. React gets around this with the context hook and which you can access implicitly if it has been injected at a higher level. Do you know if Zig supports something of the sort?

It doesn't and likely never will.

This has been a non-issue for years with Allocator. I fail to see why it will be a problem with IO.

Re: Zig's new plan for asynchronous programs

#136

Post by [deleted] »

jayd16 wrote: post #42
woodruffw wrote: post #22

I think this design is very reasonable. However, I find Zig's explanation of it pretty confusing: they've taken pains to emphasize that it solves the function coloring problem, which it doesn't: it pushes I/O into an effect type, which essentially behaves as a token that callers need to retain. This is a form of coloring, albeit one that's much more ergonomic. (To my understanding this is pretty similar to how Go sol…

Actually it seems like they just colored everything async and you pick whether you have worker threads or not. I do wonder if there's more magic to it than that because it's not like that isn't trivially possible in other languages. The issue is it's actually a huge foot gun when you mix things like this. For example your code can run fine synchronously but will deadlock asynchronously because you don't account for m…

[deleted]

Re: Zig's new plan for asynchronous programs

#137

Post by woodruffw »

throwawaymaths wrote: post #133
woodruffw wrote: post #22

I think this design is very reasonable. However, I find Zig's explanation of it pretty confusing: they've taken pains to emphasize that it solves the function coloring problem, which it doesn't: it pushes I/O into an effect type, which essentially behaves as a token that callers need to retain. This is a form of coloring, albeit one that's much more ergonomic. (To my understanding this is pretty similar to how Go sol…

1) zig's io is not a viral effect type, you can in principle declare a global io variable and use it everywhere that any library calls for it. Not best practice for a library writer, but if you're building an app, do what you want. 2) There are two things here, there is function coloring and the function coloring problem. The function coloring problem is five things: https://journal.stuffwithstuff.com/2015/02/01/what…

> 1) zig's io is not an effect type, you can in principle declare a global io variable and use it everywhere that any library calls for it.

That's an effect, akin to globally intermediated I/O in a managed runtime.

To make it intuitive: if you have a global token for I/O, does your concurrent program need to synchronize on it in order to operate soundly? Do programs that fail to obtain the token behave correctly?

Re: Zig's new plan for asynchronous programs

#138

Post by debugnik »

iviv wrote: post #95
debugnik wrote: post #29

Earlier quoted context omitted.

And I bet those green threads still need an IO type of some sort to encode anything non-pure, plus usually do-syntax. Comparing merely concurrent computations to I/O-async is just weird. In fact, I suspect that even those green threads already have a "colourful" type, although I can't check right now.

Pure actions can be run in parallel with https://hackage-content.haskell.org/package/parallel/docs/Co... Impure actions use the IO monad like always in Haskell: https://hackage.haskell.org/package/base-4.21.0.0/docs/Contr... (or the higher-level async library) I suppose an extreme version of the function coloring argument could be that all types are colors.

In a sense, kinda? The function colour problem is that you can't call an async API from a non-async caller without modifying the entire call chain (or blocking the full thread). In async/await languages the conflict comes from changing the return types; the syntax is orthogonal.

Maybe in practice some properties of code are better off being whole-program than represented as types, even if they're less modular or harder to check.

Also thanks for the reference, I haven't touched Haskell in ages; I'm more of an F# and OCaml guy.

Re: Zig's new plan for asynchronous programs

#139

Post by int_19h »

bevr1337 wrote: post #112
bloppe wrote: post #92

Earlier quoted context omitted.

How are the tradeoffs meaningfully different? Imagine that, instead of passing an `Io` object around, you just had to add an `async` keyword to the function, and that was simply syntactic sugar for an implied `Io` argument, and you could use an `await` keyword as syntactic sugar to pass whatever `Io` object the caller has to the callee. I don't see how that's not the exact same situation.

In the JS example, a synchronous function cannot poll the result of a Promise. This is meaningfully different when implementing loops and streams. Ex, game loop, an animation frame, polling a stream. A great example is React Suspense. To suspend a component, the render function throws a Promise. To trigger a parent Error Boundary, the render function throws an error. To resume a component, the render function returns…

.NET has promises that you can poll synchronously. The problem with them is that if you have a single thread, then by definition while your synchronous code is running, none of the async callbacks can be running. So if you poll a Task and it's not complete yet, there's nothing you can do to wait for its completion.

Well, technically you can run a nested event loop, I guess. But that's such a heavy sync-wrapping-async solution that it's rarely used other than as a temporary hack in legacy code.

Re: Zig's new plan for asynchronous programs

#140

Post by matu3ba »

zamalek wrote: post #102
SkiFire13 wrote: post #85

Earlier quoted context omitted.

The function coloring problem actually comes up when you implement the async part using stackless coroutines (e.g. in Rust) or callbacks (e.g. in Javascript). Zig's new I/O does neither of those for now, so hence why it doesn't suffer from it, but at the same time it didn't "solve" the problem, it just sidestepped it by providing an implementation that has similar features but not exactly the same tradeoffs.

It's sans-io at the language level, I like the concept. So I did a bit of research into how this works in Zig under the hood, in terms of compilation. First things first, Zig does compile async fns to a state machine: https://github.com/ziglang/zig/issues/23446 The compiler decides at compile time which color to compile the function as (potentially both). That's a neat idea, but... https://github.com/ziglang/zig/issu…

> I still think sans-io at the language level might be the future, but this isn't a complete solution. Maybe we should be simply compiling all fns to state machines (with the Rust polling implementation detail, a sans-io interface could be used to make such functions trivially sync - just do the syscall and return a completed future).

Can you be more specific what is missing in sans-io with explicit state machine for static and dynamic analysis would not be a complete solution? Serializing the state machine sounds excellent for static and dynamic analysis. I'd guess the debugging infrastructure for optimization passes and run-time debugging are missing or is there more?

Post reply on HN