Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Portable and Interoperable Async Rust

ncameron.org

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Portable and Interoperable Async Rust

#61

Post by KingOfCoders »

zaphar wrote: post #32
KingOfCoders wrote: post #28

Earlier quoted context omitted.

I only have rudimentary knowledge of Golang (but think the blocked/green automatic scheduling is excellent). How does go nest aync calls? func f() { } func g() { } func h() { go g() go f() } What happens on f() Are the g() and f() calls inside h() blocking? Or are they async and the block happens at the point of return? Which would be the main difference to languages with an async keyword, were you need to be explici…

The go keyword executes the called function asynchronously so g() and f() won't block h(). If you need a computed result from g() or f() then you'll need to use a channel or a shared mutex guarded value to get it. A channel is the correct default choice and the mutex should only be used if you need it for performance or other reasons.

I understood from the OP that in Golang the sync and async code would be the same - contrary to e.g. Rust were you have async/wait. Go achieves this with coroutines and the go keyword.

f()

is a sync call to the function f, and the function f used async calls inside. Somewhere then needs to be a transition from async to sync contexts (aka wait/block).

I wondered where this happens.

From your comment I assume there is a difference, in sync code I would do

x = f()

while in async code I would use

f(channel)

?

Re: Portable and Interoperable Async Rust

#62

Post by zackmorris »

Zababa wrote: post #60
mamcx wrote: post #53

Earlier quoted context omitted.

Rust is hard at the start, but easy after. But I feel async keep it hard. The sad part is that async is SO infectious that you are forced to move all on it to align with the rest of the ecosystem. I also believe the way all of this is presented is not the right abstraction. Actors + CSP is probably the best way. Plus, even if concurrency parallelism I think the parallelism idioms make more sense (pin to the "thread",…

> The sad part is that async is SO infectious that you are forced to move all on it to align with the rest of the ecosystem. That's the problem with monadic stuff in general. One solution to that might be to keep the async part on the "edge" of your programs (a bit like the functional core, imperative shell pattern or the hexagonal architecture), write all your logic without async and use async only on the edge.

Thanks for that, I wasn't familiar with hexagonal architecture.

I think there's a fundamental concept here about dealing with IO in a pure functional programming (FP). For me, stuff like monads make reasoning about IO in FP languages like Haskell really difficult.

But I haven't really encountered that difficulty with ClojureScript. It pauses and resumes endlessly alongside Javascript, and uses that stop-the-world mechanism to provide and accept data for IO without using monads. So we can write all of the pure functional ClojureScript we want, blissfully unaware that monads even exist. Whereas, other FP languages seem to think of IO as this thing that happens while your program is running, and get lost in the weeds.

Where this is important is for static analysis. Without mutability, we can take the whole syntax tree and turn it into intermediate code (I-code) and transform that tree in all kinds of fun ways with concepts from Lisp. But once we have a mutable variable, that entry/exit point of the logic has to be carried along like an imaginary number, which creates forks in the road that are more difficult to analyze because every fork doubles the analysis required, which eventually leads to an explosion of complexity that limits how far we can optimize or even understand imperative programming (IP) languages.

Now imagine an IP language like C, with its myriad of mutable variables on almost every line. If we transpiled that to an FP language, we'd see countless entry/exit points around pure functional code, with intractable complexity around the mutable state stored in the variables. To the point that it can't really be statically analyzed. Then we get excited about fractional improvements in performance, without realizing that we missed out on orders of magnitude higher gains with parallelization and other transformations that could have happened.

To me, once programmers see this, they can't really unsee it. Our whole world is built on imperative code that we just don't understand. And I am starting to feel that this mutable/monadic/async behavior (whatever we want to call it) is an anti-pattern. We should be trying to get to programming that works more like a spreadsheet, where we can play with the inputs and see the results of the logic in real time without side effects.

Re: Portable and Interoperable Async Rust

#63

Post by Zababa »

zackmorris wrote: post #62
Zababa wrote: post #60

Earlier quoted context omitted.

> The sad part is that async is SO infectious that you are forced to move all on it to align with the rest of the ecosystem. That's the problem with monadic stuff in general. One solution to that might be to keep the async part on the "edge" of your programs (a bit like the functional core, imperative shell pattern or the hexagonal architecture), write all your logic without async and use async only on the edge.

Thanks for that, I wasn't familiar with hexagonal architecture. I think there's a fundamental concept here about dealing with IO in a pure functional programming (FP). For me, stuff like monads make reasoning about IO in FP languages like Haskell really difficult. But I haven't really encountered that difficulty with ClojureScript. It pauses and resumes endlessly alongside Javascript, and uses that stop-the-world mec…

How does Clojure deals with asynchronous code? In OCaml, you can do IO pretty much everywhere you want, but as soon as you want asynchronous code, you have to use monadic code that will infect everything you use. It's also known as "function coloring" in JavaScript. Having one async part in your code will tend to make everything async, so the best way to tame that is to keep async stuff (which tends to be IO) at the edge of the program. Or be like Go and have preemptive multitasking with "transparent" blocking, where you can write regular code and have everything work asynchronously.

Re: Portable and Interoperable Async Rust

#64

Post by zaphar »

KingOfCoders wrote: post #61
zaphar wrote: post #32

Earlier quoted context omitted.

The go keyword executes the called function asynchronously so g() and f() won't block h(). If you need a computed result from g() or f() then you'll need to use a channel or a shared mutex guarded value to get it. A channel is the correct default choice and the mutex should only be used if you need it for performance or other reasons.

I understood from the OP that in Golang the sync and async code would be the same - contrary to e.g. Rust were you have async/wait. Go achieves this with coroutines and the go keyword. f() is a sync call to the function f, and the function f used async calls inside. Somewhere then needs to be a transition from async to sync contexts (aka wait/block). I wondered where this happens. From your comment I assume there is…

Close. If I execute say want to run some function asynchronously I use the go keyword to execute it. But I can't get a return value if I do that so I need some other mechanism to get the return value. One way is to pass a channel into the function and expect the function to return the value to me via that channel. like so:

    func f(ch chan[int]) {
     ch
then I can call that function asynchronously

    go f(ch)
and later when I want the value from f I can retrieve it from the channel

    i :=
The net effect of the all the above is that async and non async code is highly composable. if I have a function that computes a value and I want to get that value asynchronously then I can wrap it in a function that uses a channel to get the value to me.

    go func() { ch
Every function is a potential asynchronous function.

Re: Portable and Interoperable Async Rust

#65

Post by zackmorris »

Zababa wrote: post #63
zackmorris wrote: post #62

Earlier quoted context omitted.

Thanks for that, I wasn't familiar with hexagonal architecture. I think there's a fundamental concept here about dealing with IO in a pure functional programming (FP). For me, stuff like monads make reasoning about IO in FP languages like Haskell really difficult. But I haven't really encountered that difficulty with ClojureScript. It pauses and resumes endlessly alongside Javascript, and uses that stop-the-world mec…

How does Clojure deals with asynchronous code? In OCaml, you can do IO pretty much everywhere you want, but as soon as you want asynchronous code, you have to use monadic code that will infect everything you use. It's also known as "function coloring" in JavaScript. Having one async part in your code will tend to make everything async, so the best way to tame that is to keep async stuff (which tends to be IO) at the…

You know, I wasn't entirely sure, but after researching it, the "let" form in Clojure is a monad.

Monads are something that I keep trying to learn, but for whatever reason, the info just won't stick. After decades of doing this, my brain automatically seeks out the laziest way of doing things (while still being deterministic, testable, automatable, etc). Monads seem to be a very "hands on" way of doing FP programming, which to me defeats the whole purpose. I would probably only use them in an emergency, or to port existing functionality from an imperative language, like I mentioned.

These are the first 3 links that popped up for my Google context:

https://github.com/khinsen/monads-in-clojure/blob/master/PAR...

https://cuddly-octo-palm-tree.com/posts/2021-10-03-monads-cl...

https://functionalhuman.medium.com/functional-programing-wit...

Aspects of this do look eerily similar to async (promises/futures), like maybe monads could be implemented via nullable/optional values. I think of promises as polling a nonblocking stream result until the point in the code where the result is needed, and then blocking until the promise is fulfilled. Which is basically fork/join of threads of execution, with different syntax.

The articles mention that Haskell has syntactic support (I assume sugar) for monads. I'm nearly always against syntactic sugar and domain-specific languages (DSL) though, because they obfuscate what's really going on and double the mental load by creating two or more ways of doing the same thing. It would be fine if languages let us instantly reformat the code by transpiling with various languages features toggled (like Go's gofmt but more than just whitespace) so we could see what the syntactic sugar is doing. But nobody does anything like that, which is why I'm skeptical.

I feel like monads are one way of approaching mutability, but there are others. I'm curious how shadowing variables and even stuff like Rust's borrow checker plays into this. Like why couldn't we have a pure FP language with only immutable data and no borrow checker? That executes in its entirety when new data arrives on a queue like STDIN or a queue like STDOUT has a slot available, otherwise it blocks? I guess fundamentally, I don't understand why a spreadsheet needs scripting (written in mutable languages of all things!) or FP needs monads.

Another insight is that a monad isn't really an optional value, it's a way of executing multiple potential branches of logic. Which is similar to electrical circuits or switching at railway stations. This happens in shaders when both sides of a branch are executed, but only the outcome that matches the result of the branch is kept:

https://fsharpforfunandprofit.com/rop/

https://vimeo.com/113707214

Re: Portable and Interoperable Async Rust

#66

Post by mamcx »

Zababa wrote: post #60
mamcx wrote: post #53

Earlier quoted context omitted.

Rust is hard at the start, but easy after. But I feel async keep it hard. The sad part is that async is SO infectious that you are forced to move all on it to align with the rest of the ecosystem. I also believe the way all of this is presented is not the right abstraction. Actors + CSP is probably the best way. Plus, even if concurrency parallelism I think the parallelism idioms make more sense (pin to the "thread",…

> The sad part is that async is SO infectious that you are forced to move all on it to align with the rest of the ecosystem. That's the problem with monadic stuff in general. One solution to that might be to keep the async part on the "edge" of your programs (a bit like the functional core, imperative shell pattern or the hexagonal architecture), write all your logic without async and use async only on the edge.

Is not that simple (in Rust?).

DB interfacing is pretty deep in the chain so you can't avoid it (without re-implement what sql do already).

Re: Portable and Interoperable Async Rust

#67

Post by nyberg »

skohan wrote: post #38
c-cube wrote: post #33

Earlier quoted context omitted.

Zig has tagged unions and the corresponding switch expressions, is that not equivalent?

I was not aware of these, either it was added since I kicked the tires on Zig or I just wasn't aware of it, but yes it looks pretty good! The one drawback I see is that it seems a bit verbose: i.e. the tag set itself has to be declared as a separate enum, and then the tags need to be repeated inside the union. So it looks to be slightly bolted-on and unergonomic (similar to TypeScript's implementation) but I haven't…

You can use union(enum) to avoid having to write a separate enum definition for tags and std.meta.FieldEnum() exists should you also want to derive an enum from the union later on.

Re: Portable and Interoperable Async Rust

#68

Post by KingOfCoders »

zaphar wrote: post #64
KingOfCoders wrote: post #61

Earlier quoted context omitted.

I understood from the OP that in Golang the sync and async code would be the same - contrary to e.g. Rust were you have async/wait. Go achieves this with coroutines and the go keyword. f() is a sync call to the function f, and the function f used async calls inside. Somewhere then needs to be a transition from async to sync contexts (aka wait/block). I wondered where this happens. From your comment I assume there is…

Close. If I execute say want to run some function asynchronously I use the go keyword to execute it. But I can't get a return value if I do that so I need some other mechanism to get the return value. One way is to pass a channel into the function and expect the function to return the value to me via that channel. like so: func f(ch chan[int]) { ch then I can call that function asynchronously go f(ch) and later when…

Thanks a lot! Can only upvote you once sadly.

Re: Portable and Interoperable Async Rust

#69

Post by fulafel »

tcoff91 wrote: post #4
camdenlock wrote: post #2

Why has Rust struggled so much with this, where Go has succeeded from the start with its language-level “goroutine” concept and runtime? Maybe it just wasn’t a focal area for the original Rust designers?

Go is a very different language than rust. Go has automatic memory management & garbage collection. This automatically disqualifies it from being used in many scenarios that rust is designed to support, like embedded systems. Go’s runtime model just makes stuff like this vastly simpler. Rust can’t impose the same kind of runtime model that go has.

Lots of embedded systems are very happy using garbage collection and languages with runtimes.

Re: Portable and Interoperable Async Rust

#70

Post by KingOfCoders »

zaphar wrote: post #64
KingOfCoders wrote: post #61

Earlier quoted context omitted.

I understood from the OP that in Golang the sync and async code would be the same - contrary to e.g. Rust were you have async/wait. Go achieves this with coroutines and the go keyword. f() is a sync call to the function f, and the function f used async calls inside. Somewhere then needs to be a transition from async to sync contexts (aka wait/block). I wondered where this happens. From your comment I assume there is…

Close. If I execute say want to run some function asynchronously I use the go keyword to execute it. But I can't get a return value if I do that so I need some other mechanism to get the return value. One way is to pass a channel into the function and expect the function to return the value to me via that channel. like so: func f(ch chan[int]) { ch then I can call that function asynchronously go f(ch) and later when…

"The net effect of the all the above is that async and non async code is highly composable."

How does this differ from Rust (Or Typescript etc.) where we would use

  async f() -> i32 { }

  fn g() { f().wait() }
to block?
Post reply on HN