Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async and Await in Rust: a full proposal

boats.gitlab.io

Post reply on HN
Print view View original

111–120 of 196 posts

Re: Async and Await in Rust: a full proposal

#111

Post by catern »

gpderetta wrote: post #81
shawn wrote: post #74

Earlier quoted context omitted.

Undelimited continuations are memory sieves. Delimited continuations are better but a bit less useful.

Genuine question: aren't all one shot continuations (like the ones this thread is about) naturally delimited? Is there even such a thing as an undelimited one shot continuation?

Sure there is: take an undelimited continuation API, and restrict the resulting continuations to be called only once. Most use cases still work just fine, and I think it would be strange to call that anything other than "undelimited one-shot continuations".

Maybe what you have in mind, though, is the traditional argument, that there's no such thing as undelimited continuations. All continuations are naturally "delimited" by the boundary of the language runtime, operating system, etc, even with undelimited, multi-shot continuations. One-shot continuations make this argument more natural, because they have an obvious stack-based implementation for both delimited and undelimited continuations, and stacks are clearly delimited.

Re: Async and Await in Rust: a full proposal

#112

Post by steveklabnik »

kibwen wrote: post #109
lapinot wrote: post #105

Earlier quoted context omitted.

Yeah, sure, i understand that, hn comments just need to be a bit spicy (btw, any ref of papers or other stuff in that area? i didn't follow closely but didn't see too much moving, actually i don't even know what the big question marks are). Although i really believe it's not about if but how, so let's hope that when the time comes Rust will be able to take the tough changes necessary to unify the different branches t…

For the moment, you can see the progress on empowering the type system in the RFC for generic associated types, which is intended both to be the simplest extension to the type system that allows for some important patterns (e.g. streaming iterators and collection traits) and to be forward-compatible with any additional extensions in the direction of HKT. Importantly, it's also intended to be intuitive for users who h…

Note also that HKT and do notation are their own separate things; you can have HKT without do notation. That's all HKT stuff, not do notation stuff.

Re: Async and Await in Rust: a full proposal

#113

Post by gpderetta »

nine_k wrote: post #85
noobermin wrote: post #27

Earlier quoted context omitted.

Alright, so I do computational plasma physics simulations for a living. Given our problem sizes often reach sizes which require teraflop level of parallelism to get done before I retire (that's my scale, I'm sure you're aware of petaflop sims the fluid/engineering guys do) we do need concurrency but it's done by chopping up the domain, literally the volume of space we simulate, onto processors. Now, we could possibly…

At worst, you'll have to explicitly wait for all your async calls to have completed and produce results. I suppose normally the implicit wait occurs on attempt to access the result. I also suppose you have to do it anyway in a multi-processor system, and you can't be doing it on a single core. At best, some of your CPUs would be able to run calculations for two especially fast-to-compute pieces of volume space, while…

The problem is that async adds a lot of overhead, and while the mythical sufficiently good compiler could remove it, in practice it is a lot of work for little benefit.

For those CPU bound jobs that do not fit in the classical openmp style scheduling, more dynamic async style scheduling might be appropriate (cilk style work stealing for example) but the async granularity is hardly ever the function boundary.

Re: Async and Await in Rust: a full proposal

#114

Post by xfer »

anko wrote: post #58
millstone wrote: post #37

Earlier quoted context omitted.

Why would kernel threads be polling in practice? Won't most be waiting on locks, file descriptors, timers, etc?

You're right, in theory, if your select() or epoll() doesn't have a timeout. And the threads aren't doing any active work. But if you're optimising for that state, then you don't care about inefficiencies of one model or another. There are also costs in setting and checking those locks etc. Sure, you can build solutions to optimise a broken model, which we've done over decades (with quite a bit of success!), but it d…

> But if you're optimising for that state, then you don't care about inefficiencies of one model or another.

Sure you do, there is noticable memory and ctx switching overhead(for large no. of them ofc) in stackful coroutines.

Re: Async and Await in Rust: a full proposal

#115

Post by gpderetta »

Matthias247 wrote: post #55
UncleEntity wrote: post #21

Earlier quoted context omitted.

Why would anyone assume that a GC is needed for async/await? The Future object seems to do the all bookkeeping needed by the borrow checker (which I believe is what rust uses to track the lifetime of objects).

Maybe because most future implementations (e.g. in Javascript, the proposed C++ solution, Seastar, etc) work by scheduling a continuation on an event loop. Which obviously requires at least some form of dynamic memory for queuing up these continuations.

I think seastar futures are allocation free. Standard c++ futures are not really a paragon of efficiency or good design. Still no GC though.

Re: Async and Await in Rust: a full proposal

#116

Post by withoutboats »

lapinot wrote: post #101

And yet we somehow don't acknowledge the fact that this is just do-notation for some specific monad--yeah, that powerful abstraction that can't be expressed in Rust because we don't allow higher-order polymorphism. Don't get me wrong, i'm bitter because i feel like Rust really is almost in the right direction for the future of language design. Yet there is a long time before we get a language with a really precise ty…

I don't have time to write a whole essay, so let me just establish my credentials:

- I wrote the generic associated types RFC (how Rust will implement higher kinded polymorphism).

- I wrote the const generics RFC (the closest Rust will get to dependent types).

- I wrote the async/await RFC, as well as the linked blog post.

That is to say that I am intimately familiar with how Rust's type system can be extended to support more "powerful" abstractions.

Monads as implemented in pure functional programming languages like Haskell cannot usefully abstract over asynchronous and synchronous IO in Rust for a variety of reasons having to do with the way the type system exposes low level details by virtue of Rust being a systems programming language. I do not believe that `do` notation could be a useful mechanism for achieving either the ergonomics or the performance that async/await syntax will have in Rust.

I'm responding to you because you're the top comment, but I could write a similar response to a lot of comments here. Monads, stackful coroutines, green threads, CSP, etc - we've heard of them! :) We have well-motivated reasons to choose async/await: its the only solution that meets our requirements.

Re: Async and Await in Rust: a full proposal

#117

Post by chongli »

steveklabnik wrote: post #112
kibwen wrote: post #109

Earlier quoted context omitted.

For the moment, you can see the progress on empowering the type system in the RFC for generic associated types, which is intended both to be the simplest extension to the type system that allows for some important patterns (e.g. streaming iterators and collection traits) and to be forward-compatible with any additional extensions in the direction of HKT. Importantly, it's also intended to be intuitive for users who h…

Note also that HKT and do notation are their own separate things; you can have HKT without do notation. That's all HKT stuff, not do notation stuff.

If you have HKT then isn't do notation a piece of cake? It just desugars into monadic bind and return operators. Or is there something specific to rust that would mess with that?

Re: Async and Await in Rust: a full proposal

#118

Post by pcwalton »

chongli wrote: post #117
steveklabnik wrote: post #112

Earlier quoted context omitted.

Note also that HKT and do notation are their own separate things; you can have HKT without do notation. That's all HKT stuff, not do notation stuff.

If you have HKT then isn't do notation a piece of cake? It just desugars into monadic bind and return operators. Or is there something specific to rust that would mess with that?

Imperative control flow, for one. Think about how you would implement "break" that can break out of multiple nested loops in do-notation…

Re: Async and Await in Rust: a full proposal

#119

Post by steveklabnik »

chongli wrote: post #117
steveklabnik wrote: post #112

Earlier quoted context omitted.

Note also that HKT and do notation are their own separate things; you can have HKT without do notation. That's all HKT stuff, not do notation stuff.

If you have HKT then isn't do notation a piece of cake? It just desugars into monadic bind and return operators. Or is there something specific to rust that would mess with that?

Please see my comment here: https://news.ycombinator.com/item?id=17537878

Re: Async and Await in Rust: a full proposal

#120

Post by withoutboats »

gpderetta wrote: post #71

I'm somewhat disappointed that rust is going with async style stackless continuations. I'm a huge fan of stackfull coroutines/continuations as they are much more elegant and flexible. The downside is that they need a full stack, but I strongly believe (but can't prove) that rust has enough annotations and lifetime capabilities that it should be able to guarantee single frame allocation (or even no allocation for full…

You emphasize that stackfull coroutines could get down to a single allocation, but I'm not sure exactly what you mean. I could trivially get a stackfull coroutine down to a single allocation, the same way that systems thread do it: by allocating a defined stack size and raising a seg fault when you go over it.

The trick that "stackless coroutines" can achieve, and do achieve in Rust, is not just getting a single allocation, but getting a single perfectly sized alloc every time. It is exactly big enough to hold all the data it could ever need to hold, no more no less. You mention we have a lot of annotations, but the annotations that I know of that can achieve that are annotating every yield point so we can compute the stack space we need to store at that point, which is exactly what async/await notation is for.

Post reply on HN