Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's New Async I/O

kristoff.it

Post reply on HN
Print view View original

241–250 of 293 posts

Re: Zig's New Async I/O

#241

Post by throwawaymaths »

garaetjjte wrote: post #236

I'm confused. The trouble with "colored" functions is that they either do processing on the stack, or unwind the stack. They claim defeat of function coloring, and describe that IO implementation can use blocking/thread pool/green threads. But... these are all blocking methods, which weren't the problem in the first place! If you keep convention to never do IO using global state, you could do that practically in any…

I am not on the core team but i believe the plan is to do exactly what you are talking about, but after the API is nailed down and kinks have been ironed out by users of the existing semiblocking implementation (to possibly include the compiler), as the default LLVM coro state machine compiler has problems (for example: I think I remember Andrew complaining that it has an obligatory libc/malloc dependency).

since the new io interface has userland async/await methods, then dropping in a proper frame jumping solution will be less painful, and easier to debug, and if using coroutines proves to be challenging with the api hopefully changes to io api would be minor, versus going after stackless coroutines NOW and making large API changes often as the warts with the system uncover themselves.

Re: Zig's New Async I/O

#242

Post by gavinhoward »

audunw wrote: post #198
gavinhoward wrote: post #71

As the author of a semi-famous post about how Zig has function colors [1], I decided to read up on this. I see that blocking I/O is an option: > The most basic implementation of `Io` is one that maps to blocking I/O operations. So far, so good, but blocking I/O is not async. There is a thread pool that uses blocking I/O. Still good so far, but blocking I/O is still not async. Then there's green threads: > This implem…

It’s hard to judge before stackless coroutines are reintroduced. But I think you’re entirely wrong that the next version of it will have colored functions, even according to your definition. It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color. Changing function color could become an…

> It has been mentioned that it’s possible that the default for debug builds is that every single function is compiled as an async function. I.e. there is canonically only one function color.

But then, for those that choose to only use blocking I/O or green threads, they still pay the penalty of async merely existing.

> That’s a really bad take. It’s not much different from what they did to make allocators explicit.

I mean, Zig explicit allocators are really the same thing is Go interfaces, just dressed up as an innovative feature by a specific use case. This is what I mean by "appearing" innovative: they are taking old and tested ideas and presenting them in new ways to make them appear new.

Also, Zig could have had explicit allocators without needing to pass them to every function [1].

> They’ve moved away from designing in a bunch of fancy syntax that locks users into one particular concurrency model, and gone for a more explicit and boring design which puts power in the hands of the user.

Except that if every function is made async, they have actually removed from users the power to choose to not use async.

[1]: https://jai.community/t/context/163

Re: Zig's New Async I/O

#243

Post by Cloudef »

crabmusket wrote: post #181
Cloudef wrote: post #172

I like the IO interface simply for the fact that it would allow me to create language level vfs

Seeing the example code made me wonder if this would allow introducing capability based security. E.g. passing an `io` instance to a library which can only read a subtree of the filesystem. Edit: not quite https://news.ycombinator.com/item?id=44549430

Only if you are sure all the code uses the IO instance, if you mean language level sandboxing of untrusted code then no, zig code can always call syscalls directly. But you can compile zig to wasm which will give you capability based security.

Re: Zig's New Async I/O

#244

Post by thrwyexecbrain »

I miss the mention of boost::asio in this thread. At first glance this new Io interface feels not that dissimilar to it:

Both are generic interfaces over an event loop/executor supporting async or blocking operations. Both ship a thread-pool and a stackful coroutine backend and both can be used through their respective language's stackless coroutine implementation (co_yield in cpp and yet-unimplemented in zig).

Re: Zig's New Async I/O

#245

Post by Measter »

benreesman wrote: post #209
throwawaymaths wrote: post #202

Earlier quoted context omitted.

it's not a monad, since you can do unholy (for fp) things with it, like stash it in a struct and pass the struct around (even to functions which have no clue theres an io call) or just grab a globalized io object and use that at will arbitrarily at many entrypoints in your function. most importantly, besides the obvious situations (creating the io object, binding it to another object), it's not generally going to be…

i'm relatively confident that andrew will happily break the language again (god love him for that) when it becomes clear that you really want that algebra. though i will say, for a systems language, it's probably better to invert the lift/unlift relationship, default to do-notation and explicitly unlift into pure functions. that's almost what const meant in C++ to begin with but it lost it's way.

The same Andrew who rejected even basic interfaces in favour of duck-typed generics or manually written vtables?

Re: Zig's New Async I/O

#246

Post by n42 »

throwawaymaths wrote: post #174
n42 wrote: post #12

Earlier quoted context omitted.

Aside from the ridiculous argument that function parameters color them, the assertion that you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in

> you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in that's not true. suppose a function foo(anytype) takes a struct, and expects method bar() on the struct. you could send foo() the struct type Sync whose bar() does not use io. or you could send foo() the struct type Async whose bar uses an io stashed in the parameter , and there would be…

It's hard to parse your comment, but I think we are agreeing? I was refuting the point of the parent. you have given another example of calling an IO-taking function inside a non-IO taking function. the example I gave was initializing an IO inside the non-IO taking function. you could also, as pointed out elsewhere, use global state.

Re: Zig's New Async I/O

#247

Post by lerno »

do_not_redeem wrote: post #18
henrikl wrote: post #15

Seeing a systems language like Zig require runtime polymorphism for something as common as standard IO operations seems off to me -- why force that runtime overhead on everyone when the concrete IO implementation could be known statically in almost all practical cases?

I think it's just the Zig philosophy to care more about binary size than speed. Allocators have the same tradeoff, ArrayListUnmanaged is not generic over the allocator, so every allocation uses dynamic dispatch. In practice the overhead of allocating or writing a file will dwarf the overhead of an indirect call. Can't argue with those binary sizes. (And before anyone mentions it, devirtualization is a myth, sorry)

> care more about binary size than speed

That does not seem to be true if you look at how string formatting is implemented.

Re: Zig's New Async I/O

#248

Post by NobodyNada »

throwawaymaths wrote: post #61
do_not_redeem wrote: post #28

Earlier quoted context omitted.

Wouldn't this only work if there's only one implementation throughout the entire compliation unit? If you use 2 allocators in your app, your restricted function type has 2 possible callees for each entry, and you're back to the same problem.

> Wouldn't this only work if there's only one implementation throughout the entire compliation unit in practice how often are people using more than one io in a program?

In larger Rust applications or servers I find myself doing this very often -- for example, one application I'm working on mostly uses blocking I/O for occasional filesystem access but has a little bit of async networking.

Re: Zig's New Async I/O

#249

Post by camgunz »

Spivak wrote: post #234
camgunz wrote: post #230

Earlier quoted context omitted.

Oh no you're right it's a pain in the ass and weird. But I think it's the way because there's no good reason to have more than one event loop. Also... maybe it started out as generator wrappers? I think I read something that said that.

> no good reason to have more than one event loop Per thread—once you start working in multiple threads you have the choice to have one global event loop, which comes at the cost of all async code being effectively serialized as far as threads are concerned*, or one event loop per thread. * Which can be fine if your program is mostly not async but you have that one stubborn library. Yay async virality.

I can't say there's no good reason to have per thread event loops, but I think I can say if you do know of one you're suffering a terrible curse. I can only imagine the constraints that would force me to do this.

Re: Zig's New Async I/O

#250

Post by atombender »

tidwall wrote: post #36
ayuhito wrote: post #14

Earlier quoted context omitted.

Go also suffers from this form of “subtle coloring”. If you’re working with goroutines, you would always pass in a context parameter to handle cancellation. Many library functions also require context, which poisons the rest of your functions. Technically, you don’t have to use context for a goroutine and could stub every dependency with context.Background, but that’s very discouraged.

Context is not required in Go and I personally encourage you to avoid it. There is no shame in blazing a different path.

It's not required, but eschewing it ends up going against the grain, since so much of the ecosystem is written to use contexts, including the standard library.

For example, say you instead of contexts, you use channels for cancellation. You can have a goroutine like this:

    go func() {
      for {
        select {
        case
If you want to be able to shut this goroutine down gracefully, you're going to have an issue where http.Get() may stall for a long time, preventing the goroutine from quitting.

Likewise, processResult() may be doing stuff that cannot be aborted just by closing the stop channel. You could pass the stop channel to it, but now you're just reinventing contexts.

Of course, you can choose to only use contexts where you're forced to, and invent wrappers around standard library stuff (e.g. the HTTP client), but at that point you're going pretty far to avoid them.

I do think the context is problematic. For the purposes of cancellation, it's invasive and litters the call graph with parameters and variables. Goroutines really ought to have an implicit context inherited from its parent, since everything is using it anyway.

Contexts are wildly abused for passing data around, leading to bloated contexts and situations where you can't follow the chain of data-passing without carefully reviewing the entire call graph. I always recommend not being extremely discriminating about where to pass values in context. A core principle is that it has to be something that is so pervasive that it would be egregious to pass around explicitly, such as loggers and application-wide feature flags.

Post reply on HN