Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokio 1.0 – async runtime for Rust

tokio.rs

Post reply on HN
Print view View original

361–370 of 422 posts

Re: Tokio 1.0 – async runtime for Rust

#361

Post by GrayShade »

Animats wrote: post #326
tijsvd wrote: post #146

Earlier quoted context omitted.

I had the same thing initially. The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place. With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient. Threads, obviously, accomplish the same thing, and arguably mor…

I've been amused to watch how Rust now does a simple blocking HTTP request. A few years ago, you used the "hyper" crate, which was a convenient wrapper around the "http" crate. Now, you're supposed to use "reqwest", which is a convenient wrapper around the "hyper" crate. "Reqwest" uses the Tokio machinery, even for a blocking request. If you turn on "Trace" level logging, you can watch it start up a thread pool and g…

> A few years ago, you used the "hyper" crate, which was a convenient wrapper around the "http" crate. Now, you're supposed to use "reqwest", which is a convenient wrapper around the "hyper" crate.

That's not right. http is supposed to be a common library of types for HTTP servers and frameworks (although developers of some competing frameworks have rejected it). It was never an HTTP client like make it sound like, and it's actually newer than hyper.

As for reqwest vs. hyper, the former offers synchronous wrappers over the async ones, easier TLS support and other niceties (compression, proxy support, cookies, WASM). It's high-level and easier to use, somewhat like requests over urllib3 in the Python world.

Re: Tokio 1.0 – async runtime for Rust

#362

Post by lmm »

sagichmal wrote: post #345
lmm wrote: post #306

Earlier quoted context omitted.

> Maybe it is undesirable because that often times plain mono-thread synchronous is fast enough, easier to read, easier to debug and safe to handle to a junior ? Not everybody in a team has the same level of expertise. Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts. > And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compil…

> Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts. Writing bug-free shared memory concurrency programs with Go is trivial. Your opinion is based on outdated information.

Go's whole approach to concurrency is about avoiding sharing memory; channels suspend in much the same way as async/await yield points.

Re: Tokio 1.0 – async runtime for Rust

#363

Post by pjmlp »

ragnese wrote: post #349
skohan wrote: post #336

Earlier quoted context omitted.

> it's a choice made by a particular external library Yeah I think it points to a culture problem. In some ways because dependency management is so easy with Cargo, I think it creates the temptation to just throw in some dependencies to make something work without truly understanding the overall complexity of what you're creating. Something very similar happens in the NodeJS world. > it splits the ecosystem This is s…

Agreed with your point about Cargo. It's a double edged sword. We absolutely have an NPM/leftpad culture in Rust. Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.

Dependencies are relatively easy, actually. Just most don't bother learning how to do it, and do it PHP style with header includes.

On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box.

Plus vendoring C and C++ libraries is not a dark science, only known by old druids.

Re: Tokio 1.0 – async runtime for Rust

#364

Post by littlestymaar »

simias wrote: post #77

I don't really get these modern async APIs. In languages like Javascript I thought they only made sense because JS interpreters are (historically) single-threaded, so you really have no choice but async to express some concepts. Fine. But in Rust you can just spawn threads, share data through channels or mutexes, use OS-provided async IO primitives to poll file descriptors and do event-driven programming etc... I tri…

> you can just spawn threads, share data through channels or mutexes, use OS-provided async IO primitives to poll file descriptors and do event-driven programming […] > I have to take an aspirin every time I need to dig into async-heavy code.

I've seen that a lot on the internet and I guess it must depend where you come from, because I find async/await orders of magnitude easier to reason about than threads+channel (and I'm not even talking about using epoll manually, which is just inscrutable as soon as there is a little bit of complexity involved)

Re: Tokio 1.0 – async runtime for Rust

#365

Post by sorenbs »

sanxiyn wrote: post #53
sorenbs wrote: post #41

Is it a concern that both Tokio and stdlib have AsyncRead+AsyncWrite traits that seem to be incompatible?

Yes, it is a concern, but Tokio people decided transition can happen without breaking changes hence it does not block 1.0. See https://github.com/tokio-rs/tokio/issues/2716 for details.

Thank you!

Re: Tokio 1.0 – async runtime for Rust

#366

Post by skohan »

pjmlp wrote: post #363
ragnese wrote: post #349

Earlier quoted context omitted.

Agreed with your point about Cargo. It's a double edged sword. We absolutely have an NPM/leftpad culture in Rust. Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.

Dependencies are relatively easy, actually. Just most don't bother learning how to do it, and do it PHP style with header includes. On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box. Plus vendoring C and C++ libraries is not a dark science, only known by old druids.

Maybe I've just missed it, but I have found pkg-config difficult to use and poorly documented. It's fine if you are installing things with a package manager, but I found it took some trail-and-error to figure out how to do this for my own lbraries, or for things built manually from source.

Also with c/c++ style system dependencies, I feel like there are a lot of issues with things like version conflicts which are solved much more simply by a package manager like cargo.

I agree that it's functional, but to say relatively easy I think is a bit of a stretch.

Re: Tokio 1.0 – async runtime for Rust

#367

Post by steveklabnik »

secondcoming wrote: post #358
steveklabnik wrote: post #260

Earlier quoted context omitted.

What about this bugs you? You know that it’s not like, a busy loop calling poll all the time, right?

Well, I took the phrase 'repeatedly call' to mean pretty much that! What bugs me about it, if I'm understanding it correctly, is that you have two options once an async call is issued: - the calling thread effectively waits for completion. This is fine if a fork/join pattern is useful to you (i.e. issue N async calls and then wait for N completions). This isn't proper asynchrony though. - the future is pushed on to a…

I can't speak to the same level of depth about the C++ model as the Rust one, but, while you could do those things, it's not the usual way that it works, at least, if I'm understanding your terms correctly. I'll admit that I find your terms in the first bullet pretty confusing, and the second, only slightly. Let's back up slightly. You have:

* A future. You can call poll on a future, and it will return you either "not yet" or "done." This API is provided by the standard library. You can create futures with async/await as well, which is provided by the language. These tend to nest, so you can end up with one big future that's composed out of smaller futures.

* A task. Tasks are futures that are being executed, rather than being constructed. Creating a task out of a future may place the future on the heap.

* An executor. This is provided by Tokio. By handing a future to Tokio's executor, you create a task. The job of the executor is to keep track of all tasks, and decide which one to call poll on next.

* A reactor. This is also provided by Tokio. An executor will often employ a reactor to help decide which task to execute and when. This is sometimes called an "event loop," and coordinates with the operating system (or, if you don't have one of those, the hardware) to know when something is ready.

* A Waker. When you call poll on a future, there's one more bit that happens we couldn't talk about until we talked about everything else. If a future is going to return "not yet," it also constructs a Waker. The Waker is the bridge between the task, the reactor, and the executor.

So. You have a task. That task needs to get something from a file descriptor in a non-blocking way. At some point, there's a future way down in the chain whose job it is to handle the file descriptor. When you ask it to be created, it will return "not ready", and construct a waker that uses epoll (or whatever) via the reactor. At some point, the data will be ready, and the reactor will notice, and tell the executor "hey this task is now ready to execute again," and when some time is free, the executor will eventually call poll on it a second time. But until that point, the executor knows it's not ready, and so won't call poll again.

Whew. Does that make sense? I linked my talks in this thread already, but this is kind of a re-hash of them.

Re: Tokio 1.0 – async runtime for Rust

#368

Post by pm215 »

ibraheemdev wrote: post #133
sanxiyn wrote: post #121

Earlier quoted context omitted.

It should, but it shouldn't be the default. Right now, reqwest is the default HTTP request library in Rust ecosystem, and async is mandatory for reqwest. This is a bad situation to be in.

> async is mandatory for reqwest async is not mandatory for reqwest. It provides a blocking API as well. > reqwest is the default HTTP request library in Rust ecosystem There is no "default" libraries. There are popular HTTP clients other than reqwest that also provide blocking APIs such as isahc and ureq

I had a simple rust program that used reqwest -- it just pulled down a few web pages and parsed some data out of tables in the HTML. At the time reqwest had a simple synchronous API function that made this easy. The version of reqwest which added async support broke compatibility of that function and didn't appear to provide any similarly easy to use equivalent. Luckily my use-case went away (the website I was screen scraping died) so I didn't need to try to actually fix my program to work with newer reqwest versions. But it left a pretty sour taste regarding async...

Re: Tokio 1.0 – async runtime for Rust

#369

Post by zozbot234 »

sanxiyn wrote: post #129
haberman wrote: post #113

Earlier quoted context omitted.

I'm skimming your link trying to understand the design. It sounds like there is a global flag for whether the whole program is in evented or blocking mode? > during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.

Yes, it's a top level application decision, not a library decision. In fact, this is exactly how allocation and unwinding is handled in Rust.

So, it's like the choice of a Tokio executor? That's a top-level default, too.

Re: Tokio 1.0 – async runtime for Rust

#370

Post by volta87 »

> Also, Tokio would not have been possible without Aaron Turon's and Alex Crichton's early involvement.

This feels like a slap in the face for some reason.

Aaron Turon is an extremely talented individual (their PhD thesis was a landmark contributionn). They are super kind and one of the nicest human beings I've ever met. They led the Rust Project until their involvement with Tokio caused them to drop off from Tech.

Alex Crichton is an extremely talented, kind, and hyperproductive individual, who after their involvement with Tokio dropped all async/await work and luckily "refocused" on WebAssembly.

If one is going to recap the road towards Tokio 1.0 and mention all the people that have left the Rust async ecosystem or Rust all together, you might as well spell things out.

Post reply on HN