Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a shared vision for Async Rust

blog.rust-lang.org

Post reply on HN
Print view View original

81–90 of 139 posts

Re: Building a shared vision for Async Rust

#81

Post by pas »

millstone wrote: post #74
dodobirdlord wrote: post #65

Earlier quoted context omitted.

That would have to be determined at the discretion of the implementer of the library. Rust is after all capable of being used to write code that will run in a context where there is no kernel and thus no syscalls.

I don't get it - what library? I may have an `async` function and this is part of the Rust language, not tied to any library. Can I make it into a blocking function without 100% CPU?

"async/await" is just syntactic sugar for a function that returns a Future plus a state machine at yield points. You need a library (executor) to run that Future (execute that function).

The Rust standard library's block_on uses a global ThreadPool, and the docs recommend using a LocalPool if you need finer grained control.

So, to answer your question it depends on the executor (the thing that implements block_on).

Re: Building a shared vision for Async Rust

#82

Post by pas »

h0l0gr4ph1c wrote: post #78
geofft wrote: post #73

Earlier quoted context omitted.

I'm not sure exactly what you're asking, but I think it's either answered by the "What color is your function?" article I linked above, or by the answer that this is exactly why unasync exists and why I suggested that approach is worth considering, or by the answer that you can, in fact, just run the function and the "bloat" (which is just syntactic bloat - note that performance is generally going to be better!) is t…

yes sorry, those were rhetorical questions. Your point about asking you fail to see why not using a blocking executor to deal with the async code. My problem is with needing the executor at all. I must have skipped a couple of you previous pints in this thread. Apologies about that... Maybe we should start trying to think about async as being something can use if they want and ignore if they want. Code being async co…

As long as the async code doesn't depend on calling itself concurrently it should be straightforward to simply execute it on the current thread, right? (Basically using an executor that has 1 thread, the current thread.)

And it'd be great to check and optimize away all this at compile time.

Re: Building a shared vision for Async Rust

#83

Post by millstone »

adwn wrote: post #79
millstone wrote: post #76

Earlier quoted context omitted.

I agree. In Rust I write correct code slowly. "Productive" I am not. But the great strengths of Rust, like memory and thread safety, are blunted in WASM, which is already memory-safe and thread-crippled. So Rust's success in WASM must be due to other factors.

> I agree. In Rust I write correct code slowly. "Productive" I am not. I'm confused by that statement. Do you not care whether your code works correctly? Do you consider finding and fixing bugs to be separate from writing code?

Rust has lots of nonsense with zero practical benefit. Examples: PhantomData, higher-ranked trait bounds, "upstream crates may add a new impl". I have satisfied the compiler but no bugs were prevented. It's just busywork.

Re: Building a shared vision for Async Rust

#85

Post by dunkelheit »

Animats wrote: post #45
songqin wrote: post #32

Earlier quoted context omitted.

I had understood your concern as wanting to avoid the complexity of asynchronous code execution in your codebase, I did not realize your concern is about writing very low level systems code. In that case, you are doing the right thing: libraries like ureq, minreq, Isahc, curl, and more all offer what you want. It is unclear to me what you mean by keeping the community "in check". There are a lot of people who rely on…

It's not that "low level". It's that it doesn't fit the model of "mostly waiting for the network". Here are some of the things I have going on: - Incoming events UDP packets from multiple servers. These arrive at the client and go into a queue for processing. - Refreshing the 3D window. This ties up one thread almost full time. At the beginning of each frame, it reads queued events that tell it what needs to change i…

> I've previously done robotics work which had many asynchronous tasks. I've used QNX for that, and I've used ROS. There, you have a lot of intercommunicating processes, which works but has more overhead.

I wonder, why is this kind of system a bad fit for the async model? Is it because all threads must be reliably preemptable?

Re: Building a shared vision for Async Rust

#86

Post by dunkelheit »

geofft wrote: post #41
Animats wrote: post #30

Earlier quoted context omitted.

"reqwest" exposes a synchronous API, but it's doing async stuff underneath. If you turn on logging, you can see 30 or so async events associated with a single HTTP client side request. It's starting up and shutting down a polling thread just to make one HTTP client request. You must understand that most web and network related libraries will be async by default for "performance" . That's what scares me - async contam…

Let me try to rephrase this in a way that doesn't pin the blame on "async enthusiasts" as people, and see if you agree: Many years ago, well before Rust 1.0, Rust used its own M:N threading system, used segmented stacks, had it's own libuv-based event loop, etc. Also, it had garbage collection built into the language. These were removed before 1.0, which made Rust a lot better as a systems language: you could reliabl…

I think this rephrasing misses the mark a bit on the original concern. GP explicitly states that he wants Rust-the-language to remain as it is - close to the metal, no GC, with minimal runtime and 1:1 threading. The concern is indeed with the libraries/ecosystem. We are not quite there yet, but it is not hard to imagine that in a few years somebody who asks how to do some simple task in a blocking fashion will be met with replies in the vein of "well, that's not idiomatic", "why don't you use async", or "there was a library for that but it is now kind of unmaintained". All because the main effort of the community went into supporting and maintaining the async mode.

If the reqwest library indeed spawns a thread for every request, it is a good example of that dynamic. Sync mode kind of works but is clearly a second-class citizen and works in a suboptimal fashion. And if you want to peek under the hood to debug it you still have to deal with the async machinery in all its gory detail.

Re: Building a shared vision for Async Rust

#87

Post by woah »

AgalmicVentures wrote: post #37
songqin wrote: post #32

Earlier quoted context omitted.

I had understood your concern as wanting to avoid the complexity of asynchronous code execution in your codebase, I did not realize your concern is about writing very low level systems code. In that case, you are doing the right thing: libraries like ureq, minreq, Isahc, curl, and more all offer what you want. It is unclear to me what you mean by keeping the community "in check". There are a lot of people who rely on…

You're replying to John Nagle, as in https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm -- perhaps it would be worth considering his words further, before dismissing his concerns? Surely we can all agree that spinning up unnecessary threads is undesirable?

Doesn’t matter who he is, if he’s mad that libraries made for and by web and network developers are using a concurrency model that works well for their applications, he should use different libraries.

Re: Building a shared vision for Async Rust

#88

Post by woah »

millstone wrote: post #66
adwn wrote: post #63

Earlier quoted context omitted.

> what is Rust bringing to the WASM table? Unless you have to interface with a lot of mostly-header/header-only libraries written in C or C++, Rust is just so much more productive to work with than C or C++, even if you don't have to worry about memory unsafety.

How do you find Rust to be so much more productive than C++? I find C++ more productive. In C++ I mainly fight with template error messages. In Rust, I fight with typechecked generics, inserting & and '_ here or there, adding and deleting imports, refactoring to add Some and Ok - tons of nonsense housekeeping. These features have value but are quite a slog when writing code.

How are Some and Ok nonsense housekeeping? These things are fundamental parts of your program’s logic. Without them you are playing Russian roulette with runtime errors

Re: Building a shared vision for Async Rust

#89

Post by cycloptic »

ori_b wrote: post #50
cycloptic wrote: post #42

Earlier quoted context omitted.

That doesn't seem to be related to async? I don't know the details of rust's async implementation but that sounds like a problem with your application's setup -- you should be able to have a single threaded async executor that uses an event loop, or in simple cases, just calls poll/select directly? To put it another way, it's unfortunate that particular synchronous API is implemented using threads, but there is nothi…

Er? No, the point is that threads are what you want for cpu-bound tasks. Async does not deal well with long running cpu intensive jobs that hog the cpu without yield points.

Why not? The fix there sounds like it's as simple as adding some yield points.

Re: Building a shared vision for Async Rust

#90

Post by ibraheemdev »

steveklabnik wrote: post #16
andoriyu wrote: post #14

Earlier quoted context omitted.

Correctly if I'm wrong, but IIRC low-level futures are very dependent on executor? I think you actually told me that, but that was back in futures 0.1 era.

“Leaf” futures often are, yes. This term (coming from like, a tree) is a tad more descriptive than “low level” IMHO. We have yet to achieve a fully agnostic solution everywhere. That’s part of what work in this area is trying to figure out, as I understand it.

> That’s part of what work in this area is trying to figure out, as I understand it.

Yes, there have been a couple of ideas floating around, and a couple of older RFCs that needed more revision, such as boat's `#[global_executor]`, and discussion about an extension to task::Context that would allow futures to interact with their environment (timers, etc). There is a lot of work going on in this area, but nothing concrete yet.

Post reply on HN