Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokio 1.0 – async runtime for Rust

tokio.rs

Post reply on HN
Print view View original

211–220 of 422 posts

Re: Tokio 1.0 – async runtime for Rust

#211

Post by nchi3 »

pvorb wrote: post #181
pvorb wrote: post #156

Earlier quoted context omitted.

If you know that there will be 10k concurrent users in your application, go ahead and use async right from the start. But for the rest of us, simple, blocking code will do just fine and save us a few headaches.

And once you get to a limit, the alternative to rewriting everything in non-blocking code might be to put multiple instances of your blocking app on multiple VMs/k8s/whatever behind a load balancer.

Or, you just take the initial leap and write async from the start. For languages with decent abstractions such as async/await, it really isn't hard when you've done it for a while, and I'd make the same argument as one of the parent posters in that it's great "documentation".

K8s is brings way more complexity and headache, so it's kind of funny that you suggest that before using async/await.

Re: Tokio 1.0 – async runtime for Rust

#212

Post by lights0123 »

sorenbs wrote: post #41

Is it a concern that both Tokio and stdlib have AsyncRead+AsyncWrite traits that seem to be incompatible?

Rust doesn't have AsyncRead+AsyncWrite in std. You may be thinking about the third-party futures crate though, which does have a currently incompatible implementation.

Re: Tokio 1.0 – async runtime for Rust

#213

Post by jandrewrogers »

sanxiyn wrote: post #95
steveklabnik wrote: post #85

Earlier quoted context omitted.

Async/await matters in Rust specifically because the borrow checker makes writing code without it difficult, inefficient, and unergonomic: http://aturon.github.io/tech/2018/04/24/async-borrowing/ (note that some of the details have changed here, but the thrust of it is very much the same.) That said, if you can get your job done without this stuff, that's fine too, but the reasons it was pursued specifically involve…

It all comes down to "is it such a common scenario?". Honest answer is no. async is for specialized situations and shouldn't be the default.

Rust is a systems programming language. Modern server software -- a primary use case for a systems language -- is heavily async by default for a long list of compelling architectural reasons. Providing first-class language tooling to support that seems eminently sensible since this is how people will want to use the language.

When you are writing high-performance server code, async is the common scenario.

Re: Tokio 1.0 – async runtime for Rust

#214

Post by josephg »

sanxiyn wrote: post #172
efaref wrote: post #147

Earlier quoted context omitted.

How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread then you will start to queue on thread spawning. Don't forget that each thread gets its own stack taking up megabytes of memory. The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controll…

rouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer. If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading. I guess it was different in the past when computers were slow. I can believee that.

How does it handle slow http attacks? If I open 10k TCP connections to your server and drip feed http requests 1 byte at a time on each connection, what happens?

You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high enough not to trip apache's connection timeout. (So like, 20 TCP connections each sending 1 byte every 20 seconds would do it. Not sure about today but Apache used to be brought to its knees with 1 bps of bandwidth.)

You could probably mitigate this by putting nginx in front of your server, but this works because nginx uses async internally to handle requests. And that won't work if you ever do proxy passthrough (for SSE, websockets, etc).

Re: Tokio 1.0 – async runtime for Rust

#215

Post by 1f60c »

steveklabnik wrote: post #12
pfraze wrote: post #9

Earlier quoted context omitted.

Replying to myself, I found this bit in the docs explaining how Tokio decorates the main: > An async fn is used as we want to enter an asynchronous context. However, asynchronous functions must be executed by a runtime. The runtime contains the asynchronous task scheduler, provides evented I/O, timers, etc.

To provide some more color on why this isn't built in, different runtimes provide different kinds of guarantees and performance profiles. A webapp has very different requirements than an embedded system, and so we don't want to provide a single runtime. The language contains the basic things needed for the ecosystem to exist, and interoperation points, and then leaves the rest to said ecosystem. (Some of those intero…

I knew you needed to use a library like Tokio with asynchronous Rust, but I never understood why, so thanks.

That said, the last time I looked at it, Tokio was much too complex for my tastes.

Re: Tokio 1.0 – async runtime for Rust

#216

Post by secondcoming »

sanxiyn wrote: post #172
efaref wrote: post #147

Earlier quoted context omitted.

How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread then you will start to queue on thread spawning. Don't forget that each thread gets its own stack taking up megabytes of memory. The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controll…

rouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer. If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading. I guess it was different in the past when computers were slow. I can believee that.

> it handles thousands of requests per second just fine

It probably doesn't. Spawning threads per request is a lazy pattern.

We use Apache to handle billions of requests per day, and even its thread pool can be an issue.

Re: Tokio 1.0 – async runtime for Rust

#217

Post by btrask »

steveklabnik wrote: post #206
btrask wrote: post #191

Earlier quoted context omitted.

Whether it's kernel threads or green threads, the same patterns (locks, etc) are possible. Locks are supposed to be the borrow checker's bread and butter, because it can guarantee they are held before accessing shared state. But now you're saying "the borrow checker makes writing code without [async/await] difficult, inefficient, and unergonomic." I'm not saying locks are better than async/await (although they are[1]…

I am not saying that the borrow checker cannot handle locks. Locks work great. (The borrow checker does not understand locks as a special construct, to be extra clear.) Did you read the post I linked? It lays out the details. I am happy to clarify if you don’t get the specifics.

I see now, I misunderstood your original post. You were saying async/await is necessary because futures work badly, not because all the alternatives (i.e. locks) work badly.

Sorry, my mistake!

Edit to add: futures work badly in every language, so there's no shame in the borrow checker not working with them.

Edit 2: But in that case we're back to "why would Rust want async/await over (potentially green) threads with its first-class support for locks?"

Re: Tokio 1.0 – async runtime for Rust

#218

Post by jen20 »

sanxiyn wrote: post #200
btrask wrote: post #191

Earlier quoted context omitted.

Whether it's kernel threads or green threads, the same patterns (locks, etc) are possible. Locks are supposed to be the borrow checker's bread and butter, because it can guarantee they are held before accessing shared state. But now you're saying "the borrow checker makes writing code without [async/await] difficult, inefficient, and unergonomic." I'm not saying locks are better than async/await (although they are[1]…

Yes. There are some difficult technical issues. In practice, borrow checker works less well on async code than threaded code. This is partly why I prefer threading over async in Rust. Look, we went through some enormous effort to make threading good and fun again. Why wouldn't you use threading?

> Why wouldn't you use threading?

Because the C10^nK problem where n increases periodically is still a thing?

Re: Tokio 1.0 – async runtime for Rust

#219

Post by secondcoming »

oblio wrote: post #126
varajelle wrote: post #114

Earlier quoted context omitted.

> Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort Seems like that would use a lot of memory for all the stacks

Based on this, the default value is 2MB: https://unix.stackexchange.com/questions/127602/default-stac... So that would mean a lot of memory for 1 million threads, 2TB of RAM. But you can change the default. With a 64k stack you'd use up ~68GB of RAM, which doesn't seem like a lot for 1 million threads and 1 million requests happening at the same time.

How much does a 68GB cost on the cloud per day? Also, you don't have 1 million cores so quite a bit of your daily server costs will be eaten up by the OS running context switching code.

Re: Tokio 1.0 – async runtime for Rust

#220

Post by sanxiyn »

josephg wrote: post #214
sanxiyn wrote: post #172

Earlier quoted context omitted.

rouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer. If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading. I guess it was different in the past when computers were slow. I can believee that.

How does it handle slow http attacks? If I open 10k TCP connections to your server and drip feed http requests 1 byte at a time on each connection, what happens? You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high en…

Yes I use nginx. Yes I know websocket is different.
Post reply on HN