Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async-std: an async port of the Rust standard library

async.rs

Post reply on HN
Print view View original

41–50 of 238 posts

Re: Async-std: an async port of the Rust standard library

#41

Post by Argorak »

oconnor663 wrote: post #30
vbezhenar wrote: post #27

Earlier quoted context omitted.

> It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Is it 0-cost abstraction? I mean, is `sync_read` will compile to the same code like `async_read.poll`? Because turning sync into async is kind of trivial as well: just spawn new thread for that sync block.

That's a tricky question. My understanding is that while it's (in theory, modulo compiler bugs and features) a 0-cost abstraction over different underlying system APIs, those different underlying system APIs aren't necessarily the same cost. For example, if I'm trying to read from a socket in the synchronous world, I just issue the `read` system call. But in the async world I'm going to do quite a bit more: - Create…

You're pretty much correct, there is a tradeoff here. There's reasons why many high-performance systems like databases are mixed systems.

Re: Async-std: an async port of the Rust standard library

#42

Post by dnautics »

zanny wrote: post #28
dnautics wrote: post #21

Earlier quoted context omitted.

> Turning sync into async is harder in any language. Elixir's Task module (in the stdlib): future = Task.async(fn -> do_something_here end) ...do_other_things... result = Task.await(future, timeout) Mixing it with the Enum library makes concurrency dead-simple (got I a junior dev dispatching concurrent tasks in scripts with confidence), at the expense of an ugly nested double lambda. some_list_of_values |> Enum.map(f…

Does Elixer overload IO operations to be async in async contexts? Because that is largely why you cannot just wrap sync code in an async block and call it a day - once it hits a system call the thread is paused but the scheduler cannot tell that it should be dequeued. This is largely why Python async took so long to mature, because so much inbuilt functionality was making IO operations transparently using core sync i…

I'm still relatively new to the erlang vm so some of the details here might be wrong, if someone wants to correct me, please it's welcomed.

Console IO operations are actually message call to a "global group leader" which performs the operation, so they are async (and atomic). This can sometimes be confusing if an operation (such as logging) has a bunch of middlemen with an IO operation as a side effect. It's worth the atomicity, though, so none of your IO calls are interrupted by another IO call. Also, if you run a command on a remote node which dispatches IO as part of its own process, the IO will be forwarded back to its group leader (which is on your local node), which is useful for introspecting into another VM.

Disk IO is also different; each open file descriptor effectively gets its own "thread" that you send IO messages to. There are ways to bind a file descriptor directly to your current "thread", but you "have to be more careful when you do that" - you do that if performance is more important (and I have done this, it's not terrible if you are careful).

Network IO is also different; the erlang VM kind of has its own network stack, if you will, but you can set up a socket to be its own "thread" or you can bind a socket into a thread so that network packets get turned into erlang messages.

Handling blocking is all done for you by the VM, which is preemptive and tries to give threads fair share of the VM time.

When people say that programming the erlang VM is like doing everything in its own os, they aren't kidding. Except unlike linux, where your communications are basically limited, you get to interact with your processes via structured data types with coherent language (and also IPC calls are way cheaper than OS processes).

> Does Elixer overload IO operations to be async in async contexts?

Maybe the right way to answer this is: When in Elixir, presume everything is async.

Re: Async-std: an async port of the Rust standard library

#43

Post by nemothekid »

dnautics wrote: post #25
dymk wrote: post #23

Earlier quoted context omitted.

This incurs runtime overhead and boilerplate, so while it’s not as hard as other languages, it’s still harder.

like what, a few microseconds? What are you doing where you're awaiting for things in parallel where that matters? HPC? We're dispatching things that take on the order of minutes. Typically a local network request has 10-20 milliseconds of latency on our office LAN, so whatever. Clean and comprehensible code with very little boilerplate is more important when I'm reviewing my junior's code.

Well, thats a hard sell for Rust because it specifically advertises itself as a C++ replacement which means no overhead or runtime.

Re: Async-std: an async port of the Rust standard library

#44

Post by Argorak »

pohl wrote: post #8
Un1corn wrote: post #5

This remind me of the blog post "What Color is Your Function?"[0], they had to create a different library that is the same as the standard library but with async functions. I thought Rust had other, better ways to create non-blocking code so I don't understand why to use async instead. [0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

The caller of the function knows nothing about what happens within the body of the function. (Is it just doing computation, or is it doing I/O?). The async keyword is how the author of the function makes it explicit that caller should choose when to await the result. Isn't the alternative WCiYF is proposing to allow the caller to treat any function asynchronously, while having no way to discern whether doing so might…

Rust async functions are also different here. They don't run the code, they create a Future structure in it's initial state. Contrary to e.g. JavaScript, it doesn't start to run until you end up putting it on an executor. So the actual function call does something very different from what happens in other languages.

It's sometimes called "cold futures".

Re: Async-std: an async port of the Rust standard library

#46

Post by 2bitencryption »

I must be dumb, because every time I dive into async/await, I feel like I reach an epiphany about how it works, and how to use it. Then a week later I read about it again and totally lost all understanding.

What do I gain if I have code like this [0], which has a bunch of `.await?` in sequence?

I know .await != join_thread(), but doesn't execution of the current scope of code halt while it waits for the future we are `.await`-ing to complete?

I know this allows the executor to go poll other futures. But if we haven't explicitly spawned more futures concurrently, via something like task::spawn() or thread::spawn(), then there's nothing else the cpu can possible do in our process?

[0] https://github.com/async-rs/async-std/blob/master/examples/t...

Re: Async-std: an async port of the Rust standard library

#47

Post by dnautics »

nemothekid wrote: post #43
dnautics wrote: post #25

Earlier quoted context omitted.

like what, a few microseconds? What are you doing where you're awaiting for things in parallel where that matters? HPC? We're dispatching things that take on the order of minutes. Typically a local network request has 10-20 milliseconds of latency on our office LAN, so whatever. Clean and comprehensible code with very little boilerplate is more important when I'm reviewing my junior's code.

Well, thats a hard sell for Rust because it specifically advertises itself as a C++ replacement which means no overhead or runtime.

I think if you're striving for that, then a bit of complexity is warranted. Not everything has to be simple, and async is hard to do correctly without the correct abstractions. Honestly, though I was hoping Rust would go with the Actix way of doing things, but that's fine. You don't have to use Rust's async.

Re: Async-std: an async port of the Rust standard library

#48

Post by davidhyde »

Argorak wrote: post #29
davidhyde wrote: post #4

Great library, well done. In case anyone was wondering this is not a [no_std] crate even though it can be used as a replacement for std library calls. I guess it (obviously) can't be since it interfaces with the operating system so much.

Project member here. It exports stdlib types (like io::Error) where appropriate so that libraries working with these can stay compatible, so `no_std` is not really an option. The underlying library (async-task) is essentially core + liballoc, just no one made the effort to spell that out, yet.

> It exports stdlib types (like io::Error)

Excellent, good to know!

Re: Async-std: an async port of the Rust standard library

#49

Post by schrodinger »

necubi wrote: post #32
vbezhenar wrote: post #27

Earlier quoted context omitted.

> It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Is it 0-cost abstraction? I mean, is `sync_read` will compile to the same code like `async_read.poll`? Because turning sync into async is kind of trivial as well: just spawn new thread for that sync block.

Spawning a new thread for an operation is not async in the sense people typically mean. For an async IO library, you would expect it to be using async IO primitives like epoll, not just wrapping blocking operations in a thread.

That’s what I like about Go. You write sync code, but because Go routines aren’t OS threads they operate with the efficency of async code.

Re: Async-std: an async port of the Rust standard library

#50

Post by thethirdone »

Argorak wrote: post #29
davidhyde wrote: post #4

Great library, well done. In case anyone was wondering this is not a [no_std] crate even though it can be used as a replacement for std library calls. I guess it (obviously) can't be since it interfaces with the operating system so much.

Project member here. It exports stdlib types (like io::Error) where appropriate so that libraries working with these can stay compatible, so `no_std` is not really an option. The underlying library (async-task) is essentially core + liballoc, just no one made the effort to spell that out, yet.

Would there be any benefit to making a `no_std` option? I can't think of a situation you would want async std and have including std be a problem.
Post reply on HN