Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async-std: an async port of the Rust standard library

async.rs

Post reply on HN
Print view View original

21–30 of 238 posts

Re: Async-std: an async port of the Rust standard library

#21

Post by dnautics »

bryanlarsen wrote: post #11
Un1corn wrote: post #5

This remind me of the blog post "What Color is Your Function?"[0], they had to create a different library that is the same as the standard library but with async functions. I thought Rust had other, better ways to create non-blocking code so I don't understand why to use async instead. [0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

None of the 5 points in that article about callbacks in 2015 node.js apply to async in Rust. The Rust people spent years agonizing over their version of async and applied a lot of lessons learned from implementations in other languages. https://news.ycombinator.com/item?id=20676641 It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Turning sync into async is harder in…

> Turning sync into async is harder in any language.

Elixir's Task module (in the stdlib):

    future = Task.async(fn ->
      do_something_here
    end)
    
    ...do_other_things...

    result = Task.await(future, timeout)
Mixing it with the Enum library makes concurrency dead-simple (got I a junior dev dispatching concurrent tasks in scripts with confidence), at the expense of an ugly nested double lambda.

    some_list_of_values
    |> Enum.map(fn value -> 
      Task(fn -> do_something_with(value) end) 
    end)
    |> Enum.map(&Task.await(&1, timeout))

Re: Async-std: an async port of the Rust standard library

#22

Post by spullara »

rmgraham wrote: post #9
evmar wrote: post #3

In case anyone else was curious how you create nonblocking file I/O, it appears to use threads. I am curious if the number of threads is unbounded, or if they have a bounded set but accept deadlocks, or if there is a third option other than those two that I am unaware of.

Are those really the only options? I'm trying to wrap my head around how using a fixed size thread pool for I/O automatically implies deadlocks but I just can't. Unless the threads block on completion until their results are consumed instead of just notifying and then taking the next task.. I can definitely imagine blocking happening while waiting for a worker to be available, though. Did you mean simply blocking ins…

N threads, with N readers waiting for a message that will only come if the N+1 reader (still in the queue) gets a message first.

Re: Async-std: an async port of the Rust standard library

#23

Post by dymk »

dnautics wrote: post #21
bryanlarsen wrote: post #11

Earlier quoted context omitted.

None of the 5 points in that article about callbacks in 2015 node.js apply to async in Rust. The Rust people spent years agonizing over their version of async and applied a lot of lessons learned from implementations in other languages. https://news.ycombinator.com/item?id=20676641 It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Turning sync into async is harder in…

> Turning sync into async is harder in any language. Elixir's Task module (in the stdlib): future = Task.async(fn -> do_something_here end) ...do_other_things... result = Task.await(future, timeout) Mixing it with the Enum library makes concurrency dead-simple (got I a junior dev dispatching concurrent tasks in scripts with confidence), at the expense of an ugly nested double lambda. some_list_of_values |> Enum.map(f…

This incurs runtime overhead and boilerplate, so while it’s not as hard as other languages, it’s still harder.

Re: Async-std: an async port of the Rust standard library

#24

Post by DecoPerson »

otabdeveloper2 wrote: post #19
zzzcpan wrote: post #18

Earlier quoted context omitted.

The point of asynchronous programming is to know exactly where concurrency happens in your code. This both eliminates concurrency bugs and gives you predictability for high performance.

Cooperative multitasking, like it's 1995 again? No thanks.

Would you mind elaborating on your opinion here?

As far as I understand, cooperative is far more efficient than preemptive, but unsuitable for poorly written or untrusted code.

I wish to learn and would really appreciate your assistance if you are willing to help.

Re: Async-std: an async port of the Rust standard library

#25

Post by dnautics »

dymk wrote: post #23
dnautics wrote: post #21

Earlier quoted context omitted.

> Turning sync into async is harder in any language. Elixir's Task module (in the stdlib): future = Task.async(fn -> do_something_here end) ...do_other_things... result = Task.await(future, timeout) Mixing it with the Enum library makes concurrency dead-simple (got I a junior dev dispatching concurrent tasks in scripts with confidence), at the expense of an ugly nested double lambda. some_list_of_values |> Enum.map(f…

This incurs runtime overhead and boilerplate, so while it’s not as hard as other languages, it’s still harder.

like what, a few microseconds? What are you doing where you're awaiting for things in parallel where that matters? HPC? We're dispatching things that take on the order of minutes. Typically a local network request has 10-20 milliseconds of latency on our office LAN, so whatever. Clean and comprehensible code with very little boilerplate is more important when I'm reviewing my junior's code.

Re: Async-std: an async port of the Rust standard library

#26

Post by for_xyz »

dmytroi wrote: post #17
slovenlyrobot wrote: post #13

Earlier quoted context omitted.

AFAIK Windows handles truly asynchronous buffered IO in some circumstances, but I feel once you're past the point of managing the abstraction or caring about its internal details, it doesn't really matter if there is a tiny chunk of dedicated stack in the kernel, that's a problem for the OS

Yes, IOCP looks like true async I/O most of the time. While in reality it will block if file is cached, in cases if code tries to read something like 100+ MB at a time ReadFile calls can take 200ms+. So most "async I/O" frameworks have to wrap IOCP into a user space thread ...

Or disable caching for specified file when calling CreateFile API

https://docs.microsoft.com/en-us/windows/win32/api/fileapi/n...

Re: Async-std: an async port of the Rust standard library

#27

Post by vbezhenar »

bryanlarsen wrote: post #11
Un1corn wrote: post #5

This remind me of the blog post "What Color is Your Function?"[0], they had to create a different library that is the same as the standard library but with async functions. I thought Rust had other, better ways to create non-blocking code so I don't understand why to use async instead. [0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

None of the 5 points in that article about callbacks in 2015 node.js apply to async in Rust. The Rust people spent years agonizing over their version of async and applied a lot of lessons learned from implementations in other languages. https://news.ycombinator.com/item?id=20676641 It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Turning sync into async is harder in…

> It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera.

Is it 0-cost abstraction? I mean, is `sync_read` will compile to the same code like `async_read.poll`? Because turning sync into async is kind of trivial as well: just spawn new thread for that sync block.

Re: Async-std: an async port of the Rust standard library

#28

Post by zanny »

dnautics wrote: post #21
bryanlarsen wrote: post #11

Earlier quoted context omitted.

None of the 5 points in that article about callbacks in 2015 node.js apply to async in Rust. The Rust people spent years agonizing over their version of async and applied a lot of lessons learned from implementations in other languages. https://news.ycombinator.com/item?id=20676641 It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Turning sync into async is harder in…

> Turning sync into async is harder in any language. Elixir's Task module (in the stdlib): future = Task.async(fn -> do_something_here end) ...do_other_things... result = Task.await(future, timeout) Mixing it with the Enum library makes concurrency dead-simple (got I a junior dev dispatching concurrent tasks in scripts with confidence), at the expense of an ugly nested double lambda. some_list_of_values |> Enum.map(f…

Does Elixer overload IO operations to be async in async contexts? Because that is largely why you cannot just wrap sync code in an async block and call it a day - once it hits a system call the thread is paused but the scheduler cannot tell that it should be dequeued.

This is largely why Python async took so long to mature, because so much inbuilt functionality was making IO operations transparently using core sync impls that locked up any async executor.

Re: Async-std: an async port of the Rust standard library

#29

Post by Argorak »

davidhyde wrote: post #4

Great library, well done. In case anyone was wondering this is not a [no_std] crate even though it can be used as a replacement for std library calls. I guess it (obviously) can't be since it interfaces with the operating system so much.

Project member here.

It exports stdlib types (like io::Error) where appropriate so that libraries working with these can stay compatible, so `no_std` is not really an option.

The underlying library (async-task) is essentially core + liballoc, just no one made the effort to spell that out, yet.

Re: Async-std: an async port of the Rust standard library

#30

Post by oconnor663 »

vbezhenar wrote: post #27
bryanlarsen wrote: post #11

Earlier quoted context omitted.

None of the 5 points in that article about callbacks in 2015 node.js apply to async in Rust. The Rust people spent years agonizing over their version of async and applied a lot of lessons learned from implementations in other languages. https://news.ycombinator.com/item?id=20676641 It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Turning sync into async is harder in…

> It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera. Is it 0-cost abstraction? I mean, is `sync_read` will compile to the same code like `async_read.poll`? Because turning sync into async is kind of trivial as well: just spawn new thread for that sync block.

That's a tricky question. My understanding is that while it's (in theory, modulo compiler bugs and features) a 0-cost abstraction over different underlying system APIs, those different underlying system APIs aren't necessarily the same cost. For example, if I'm trying to read from a socket in the synchronous world, I just issue the `read` system call. But in the async world I'm going to do quite a bit more:

- Create an epoll descriptor.

- Add my socket to that descriptor.

- Poll the descriptor for a readiness notification.

- Read the descriptor.

Those first three system calls weren't required in the synchronous version, and unless the read is large enough to overshadow them, they represent some additional cost. But that cost is required by the OS itself, not by Rust's abstractions.

Someone with more experience writing Mio code might want to jump in and correct me here though.

Post reply on HN