Async and Await in Rust: a full proposal • Hacker News Board

Re: Async and Await in Rust: a full proposal

#141

Post by dagenix » Mon, Jul 16, 2018, 3:43 AM UTC

kibwen wrote: post #109

lapinot wrote: post #105

Earlier quoted context omitted.

Yeah, sure, i understand that, hn comments just need to be a bit spicy (btw, any ref of papers or other stuff in that area? i didn't follow closely but didn't see too much moving, actually i don't even know what the big question marks are). Although i really believe it's not about if but how, so let's hope that when the time comes Rust will be able to take the tough changes necessary to unify the different branches t…

For the moment, you can see the progress on empowering the type system in the RFC for generic associated types, which is intended both to be the simplest extension to the type system that allows for some important patterns (e.g. streaming iterators and collection traits) and to be forward-compatible with any additional extensions in the direction of HKT. Importantly, it's also intended to be intuitive for users who h…

I thought I had read at some point that generic associated types we're equivalent to HKTs in terms of what they can express. Or maybe that they were theorized to be so. Is that not the case? (Not critical, just curious)

Re: Async and Await in Rust: a full proposal

#142

Post by Matthias247 » Mon, Jul 16, 2018, 3:46 AM UTC

kuon wrote: post #135

lapinot wrote: post #101

And yet we somehow don't acknowledge the fact that this is just do-notation for some specific monad--yeah, that powerful abstraction that can't be expressed in Rust because we don't allow higher-order polymorphism. Don't get me wrong, i'm bitter because i feel like Rust really is almost in the right direction for the future of language design. Yet there is a long time before we get a language with a really precise ty…

I have been doing programming, including functional programming for more than two decades, I still don't really know what a "monad" is. Each time someone explains it to me, I understand something different.

The way I accepted it after years of confusion, and without still understand the word: A monad is any type which supports 2 specific operations. One maps an type into it's monadic form, and the other allows to apply functions to monadic types and returns the monadic type again. The latter is sometimes called bind or flatmap.

For Future types, the first thing could be called MakeReadyFuture(type), the second one Future.then(result => functionWhichReturnsANewFuture).

Re: Async and Await in Rust: a full proposal

#143

Post by wruza » Mon, Jul 16, 2018, 5:08 AM UTC

This thread is so confusing to me. Not because of complex differences in a language theory, but on choices that developers do follow due to experience in ‘the past’, regarding light threads. I understand that rust has no stdlib, that go has, and that js just doesn’t have switchable stacks. But why does almost everyone inclined to async-await in general? The answers I got in other threads (not on this exact question, but along the lines) is that async-await makes runloop-tearing points explicit. But is it really so important? How does a regular js guy manage a state incapsulation in-between their futures’ callback invocations? All the code I see is then-then-catch and it doesn’t account for, well, asynchronous effects like races; it just occurs naturally by not modifying other chains’ states. Why not just go with seamless coroutines then? Why making functions explicitly async? (Or sync, if that matters.)

Maybe I’m misunderstanding something, but since people here are so fluent in concurrent execution, can someone point me to an in-depth explanation why are light-threads, coroutines, current-continuations, etc. so opposed by async keyword (and futures in general) today?

I have some experience with low-level runlooping via coroutines in luajit and understand it to the point to be able to create asynchronous system, consisting of mix of os threads and coroutines (and it worked smoothly until our project was closed due to company’s external issues). I can say, I never felt the need of something different, neither met the ‘complexity’ of everything-can-yield rule. And the possibilities that open, i.e. scalability of simple code around io and cpu cores is just outstanding. I am genuinely curiuos what’s so great (or different) in futures, which seem to me, for now, just poor man’s light threads implemented via lexical closure overhead along with syntactic snow, running on a single-core cpu. This topic seems to be so narrow that modern google is too shallow to answer that. I believe there should be a LtU or similar thread that discusses it in classic depth. Thanks in advance!

Re: Async and Await in Rust: a full proposal

#144

Post by dpc_pw » Mon, Jul 16, 2018, 5:23 AM UTC

wruza wrote: post #143

This thread is so confusing to me. Not because of complex differences in a language theory, but on choices that developers do follow due to experience in ‘the past’, regarding light threads. I understand that rust has no stdlib, that go has, and that js just doesn’t have switchable stacks. But why does almost everyone inclined to async-await in general? The answers I got in other threads (not on this exact question,…

Note sure if I fully grasped your quesiton but here it goes.

Light threads and stackfull coroutines require stack allocation. That is their cost and it is unbearable for system-level language priding itself in zero-cost abstractions. Eg. AFAIK, it's the main reason that is making Go calling C code slow.

Also, (again AFAIK) Rust stackless coroutines and futures compile down to state machines, so I don't understand the "lexical closure overhead".

There's nothing forcing futures to be "running on a single-core cpu" in Rust, since Rust can reason about thread-safety.

Personally, after couple of months of using JS at dayjob, I very dislike JS (as I thought I would), but I love coroutines/yield. I think they will be glorious in Rust: they will allow writting reasonably nice code with an amazing performance.

Re: Async and Await in Rust: a full proposal

#145

Post by StefanKarpinski » Mon, Jul 16, 2018, 5:31 AM UTC

wruza wrote: post #143

This thread is so confusing to me. Not because of complex differences in a language theory, but on choices that developers do follow due to experience in ‘the past’, regarding light threads. I understand that rust has no stdlib, that go has, and that js just doesn’t have switchable stacks. But why does almost everyone inclined to async-await in general? The answers I got in other threads (not on this exact question,…

I have often wondered this as well and never quite found a fully satisfactory answer. I imagine that it has to do with ease of implementation, although there's an obvious and efficient implementation of making coroutines one-to-one with stacks and allowing them to yield between each other. In constrained environments like JVM and CLR, maybe that can't be done, which might explain C#'s choice of async/await. Perhaps in Rust it has to do with the interaction with the borrow checker?

Re: Async and Await in Rust: a full proposal

#146

Post by sametmax » Mon, Jul 16, 2018, 6:12 AM UTC

steveklabnik wrote: post #99

sametmax wrote: post #89

Earlier quoted context omitted.

That's an API. Like asyncio in Python is an API. If you make the event loop swapable without making this API mandatory however, it's never going to be a protocol.

The api is mandatory; it’s how async/await works.

So everybody has to implement a task for any alternative event loop and accept tasks from other implementation ?

Re: Async and Await in Rust: a full proposal

#147

Post by EGreg » Mon, Jul 16, 2018, 6:27 AM UTC

I know this may sound silly, but can someone make a quick rundown of the pros (and maybe cons) of Rust as compared to NodeJS, Go and Erlang? Why would people use it as opposed to these far more mature ecosystems, especially if it’s hard to master based on the comments I have seen from Rust users? (Not trying to be biased, actually want to ask people who do choose it.)

Re: Async and Await in Rust: a full proposal

#148

Post by grzm » Mon, Jul 16, 2018, 6:31 AM UTC

EGreg wrote: post #147

I know this may sound silly, but can someone make a quick rundown of the pros (and maybe cons) of Rust as compared to NodeJS, Go and Erlang? Why would people use it as opposed to these far more mature ecosystems, especially if it’s hard to master based on the comments I have seen from Rust users? (Not trying to be biased, actually want to ask people who do choose it.)

You might find this article useful: https://thenewstack.io/safer-future-rust/

Re: Async and Await in Rust: a full proposal

#149

Post by gpderetta » Mon, Jul 16, 2018, 6:53 AM UTC

eddyb wrote: post #134

gpderetta wrote: post #98

Earlier quoted context omitted.

I want stackfull coroutines, with custom, fast user space scheduling and task switching with guaranteed optimisation to a single stack frame and no allocation where possible. I also want the ability to convert internal iterators to internal iterators with no overhead and even (especially) if the internal iteration function has not been specifically marked (i.e. no red/blue functions). Hey, a man can dream.

> guaranteed optimisation to a single stack frame and no allocation where possible You are literally describing stackless coroutines. And the generator state transform is that optimization . If you want to get this without using generators explicitly, it's still stackless coroutines just not how Rust supports stackless coroutines. There was some discussion about making it more implicit but no progress was made in the…

Very much not. I want first class stackful continuation semantics that behave as stackless in at least all (but ideally more) scenarios where a stackless continuation would.

Re: Async and Await in Rust: a full proposal

#150

Post by gpderetta » Mon, Jul 16, 2018, 6:59 AM UTC

Matthias247 wrote: post #140

gpderetta wrote: post #115

Earlier quoted context omitted.

I think seastar futures are allocation free. Standard c++ futures are not really a paragon of efficiency or good design. Still no GC though.

I haven't read the code, but there are 2 areas where I expect allocations: - The continuation which is passed to .then, and which is typically a closure, must be type-erased, which requires an allocation. Storing the continuation in a std::function would allocate too. A short glance at https://github.com/scylladb/seastar/blob/master/core/future.... also confirms that there is a make_unique there. - Since continuation…

I do not think the continuation is type erased. I belive that then, at least optionally (depending on the actual futurator passed in) can returns a future whose type encodes the continuation type and stores it inline.

The queue might be dynamically sized which might eventually require allocation, but that can be ammortized across many futures. A large enough queue might never require reallocation.

I'm also not 100% sure as I have never used seastar.