Async-std: an async port of the Rust standard library • Hacker News Board

Re: Async-std: an async port of the Rust standard library

#211

Post by grayrest » Sat, Aug 17, 2019, 4:23 PM UTC

epage wrote: post #95

Un1corn wrote: post #5

This remind me of the blog post "What Color is Your Function?"[0], they had to create a different library that is the same as the standard library but with async functions. I thought Rust had other, better ways to create non-blocking code so I don't understand why to use async instead. [0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

This is why I'm curious about algebraic effects which was recently discussed on /r/rust [0] The main challenges I see are around usability within the language design on how best to propagate and compose them. [0] https://www.reddit.com/r/rust/comments/cjcwmu/is_there_inter...

I found out about algebraic effects a year or two ago when I ran across the efforts to bring them to OCaml. Async/await was a preliminary thing so I though algebraic effects would be a more complete solution and more in line with the Rust ethos. After asking around in some NYC meetups and on Reddit , my impression is that there isn't a lot of appetite to break additional new ground in terms of bringing fringe language features (I'm unaware of a non-academic language that features them in a production release, OCaml is closest AFAIK) into a language that already has a fairly high barrier to entry.

Re: Async-std: an async port of the Rust standard library

#212

Post by nullwasamistake » Sat, Aug 17, 2019, 4:24 PM UTC

noncoml wrote: post #193

MrBuddyCasino wrote: post #183

Earlier quoted context omitted.

Let's be real here, its not just the memory requirements, because context switching and the associated nuking of cpu caches are not free. You can go very far with it nowadays, but you can go much farther with async code, if you really need to.

Nobody is denying that async code is faster. But it’s not as dramatic as presented in the grand parent post. And IMHO the added code complexity is not worth the trouble.

We have some pretty vanilla file upload code that needs async. S3 latency is fairly high. If you're uploading a few tiny files per user per second, thread usage gets out of hand real fast.

With a simulated load of ~20 users we were running over 1000 threads.

Several posts in the chain say that 20k+ threads is "fine". Not unless you have a ton of cores. The memory and context switching overhead is gigantic. Eventually your server is doing little besides switching between threads.

We had to rewrite our s3 code to use async, now we can do many thousands of concurrent uploads no problem.

Other places we've had to use async is a proxy that intercepts certain HTTP calls and user stats uploader that calls third party analytics service.

Just sayin it's not that unusual to need async code because threading overhead is too high

Re: Async-std: an async port of the Rust standard library

#213

Post by gpderetta » Sat, Aug 17, 2019, 4:32 PM UTC

zzzcpan wrote: post #39

hathawsh wrote: post #33

Earlier quoted context omitted.

They are very similar. In cooperative multitasking, programs yield the thread to the OS, while async programs yield to the event loop. In both cases, yielding is voluntary. Are there other differences? It's probably quite easy to turn a program written one way into the other. Edit: I just remembered that in cooperative multitasking, it's probably possible for the OS to safely save the program stack pointer, meaning t…

Even conceptually the models are very different, in one control is just given up and regained unpredictably, while in the other one it is programmed, hence asynchronous programming, not multitasking.

It is cooperative, so no control is relinquished predictably. The difference is the syntactic limitations of the current async model prevent building abstractions.

Re: Async-std: an async port of the Rust standard library

#214

Post by gpderetta » Sat, Aug 17, 2019, 4:36 PM UTC

zzzcpan wrote: post #202

badamp wrote: post #123

Earlier quoted context omitted.

> in one control is just given up and regained unpredictably Which one? It’s “cooperative” ie not unpredictable. The points where one can block are predictable and documented explicitly, otherwise how would the programmer know they won’t block forever. The same should hopefully be the case for async/awaitable apis. In fact where async/await will actually give up control are harder to tease out. The differences are re…

In cooperative multitasking you can program when to give up control, not when it is regained. The regaining part is unpredictable. Which introduces a lot of non-determinism to deal with and overhead.

It is regained in exactly the same cases it would be in the async model: when a blocking operation completes and the scheduler resumes the now ready thread. As scheduler is called executor in the async world, while a thread is a coroutines, but the concepts are very similar.

Re: Async-std: an async port of the Rust standard library

#215

Post by pcwalton » Sat, Aug 17, 2019, 5:06 PM UTC

nullwasamistake wrote: post #212

noncoml wrote: post #193

Earlier quoted context omitted.

Nobody is denying that async code is faster. But it’s not as dramatic as presented in the grand parent post. And IMHO the added code complexity is not worth the trouble.

We have some pretty vanilla file upload code that needs async. S3 latency is fairly high. If you're uploading a few tiny files per user per second, thread usage gets out of hand real fast. With a simulated load of ~20 users we were running over 1000 threads. Several posts in the chain say that 20k+ threads is "fine". Not unless you have a ton of cores. The memory and context switching overhead is gigantic. Eventually…

In what language?

Re: Async-std: an async port of the Rust standard library

#216

Post by pcwalton » Sat, Aug 17, 2019, 5:07 PM UTC

zzzcpan wrote: post #201

MrBuddyCasino wrote: post #200

Earlier quoted context omitted.

> And IMHO the added code complexity is not worth the trouble. The thing is, this is just that - your opinion, generalized as The Truth. But engineering is about making the right trade-offs. Often threading will be fine, you'll win simplicity, and all is good. But sometimes you really need the performance, or your field is crowded and its a competitive advantage. Think large-scale infrastructure at AWS, central load-…

It goes deeper than that. There is plenty of research showing that shared memory multithreading is not even a viable concurrency model. The premise that threads are fine and simple is just false.

I'm not sure what you mean. One of Rust's major research contributions is to show that shared memory multithreading is a perfectly viable concurrency model, as long as you enforce ownership discipline to statically eliminate data races.

Re: Async-std: an async port of the Rust standard library

#217

Post by badamp » Sat, Aug 17, 2019, 5:19 PM UTC

zzzcpan wrote: post #202

badamp wrote: post #123

Earlier quoted context omitted.

> in one control is just given up and regained unpredictably Which one? It’s “cooperative” ie not unpredictable. The points where one can block are predictable and documented explicitly, otherwise how would the programmer know they won’t block forever. The same should hopefully be the case for async/awaitable apis. In fact where async/await will actually give up control are harder to tease out. The differences are re…

In cooperative multitasking you can program when to give up control, not when it is regained. The regaining part is unpredictable. Which introduces a lot of non-determinism to deal with and overhead.

This is no different than async/await. At some point you await a scheduled primitive, it could be a timer, io readiness, an io completion... and yield to a scheduler. You don’t specify explicitly when you return. These are not tightly coupled coroutines. This is precisely what is going on in cooperative multitasking.

I don’t see how this increases overhead to deal with either.

Basically, coop multitasking and async/await operate on the exact same execution framework, the latter just gives convenient syntactic support.

Perhaps you should see how typescript turns async await into js.

Re: Async-std: an async port of the Rust standard library

#218

Post by d0mine » Sat, Aug 17, 2019, 5:33 PM UTC

lenkite wrote: post #169

2bitencryption wrote: post #147

Earlier quoted context omitted.

Thanks. Helpful. My question is, in this example: result = await server.getStuff() second = await server.getMoreStuff(result+1) print(result) `await getStuff()` MUST terminate before `await getMoreStuff() ` begins. So this chunk alone is analagous to synchronous code, unless we're in the middle of a spawned task, and there are other spawned tasks in the executor that can be picked up.

Yes, the idea is that the thread that is executing this piece of code can "steal" other work when it is awaiting on either of those methods. Frankly, in the case of sequential flow like the above, I would rather write result = server.getStuff() second = server.getMoreStuff(result+1) print(result) and have the runtime automatically perform work-stealing for me. No need for awaits. They just litter the code. This is wh…

Go does not do that implicitly, there is an explicit "go" syntax.

Gevent in Python does something similar [implicit switching] using a dirty monkeypatching. It is great while it works. Sooner or later the explicit cooperative concurrency such as provided by async/await syntax wins (e.g., asyncio, trio, curio Python libraries)

Re: Async-std: an async port of the Rust standard library

#219

Post by therockhead » Sat, Aug 17, 2019, 5:37 PM UTC

pjmlp wrote: post #189

rubber_duck wrote: post #79

Earlier quoted context omitted.

Green threads have no place in a low level systems language like Rust whose design goals are zero cost abstraction and trivial C interop. D made a similar mistake by requiring GC/runtime from start and now even though they added ways to avoid it the ecosystem and the language design are "poisoned" by it an itmakeas it a very hard sell in some places where it could be sold as a C++ successor. Because rust made the rig…

Many systems have been developed in systems enabled GC languages. C++11 introduced a GC API in the standard library, and one of the biggest C++ game engine does use GC in their engine objects, Unreal. C++ on Windows makes heavy use of reference counting (which is a GC algorithm from CS point of view), via COM/UWP. The biggest problem to overcome is religious, not technical.

>C++ on Windows makes heavy use of reference counting (which is a GC algorithm from CS point of view), via COM/UWP.

Not sure if Ref counting is a good example here, as there is no runtime monitoring the object graph hierarchy and of course Rust it’s self uses ref counting in many situations.

Re: Async-std: an async port of the Rust standard library

#220

Post by dmytroi » Sat, Aug 17, 2019, 5:44 PM UTC

pas wrote: post #70

dmytroi wrote: post #12

Earlier quoted context omitted.

Except that io_uring is threads running in kernel. There is no true async I/O on most (if not all) current platforms - it's all threads, either in user space or in kernel space. Sometimes even deliberately, for example polling disk will give better latency compared to waiting for IRQ.

O_DIRECT + aio on Linux seems okay for preallocated files, no?

If by aio you mean Posix aio - on Linux it's implemented with user space threads and blocking I/O. Posix aio on BSD systems is implemented as kernel space thread (aio_write/etc are syscalls on BSD, and glibc functions on Linux).

If you mean io_submit, then yes, but in vast majority of cases, actual `io_submit` syscall will block, because of metadata updates, unaligned reads, etc ...