Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Asynchronous IO in Rust

medium.com

Post reply on HN
Print view View original

91–100 of 111 posts

Re: Asynchronous IO in Rust

#91

Post by acconsta »

comex wrote: post #67
theseoafs wrote: post #33

Earlier quoted context omitted.

This is backwards. Green threading can be an implementation detail, but in languages where it's a key feature, green threads let you write code that would otherwise be incorrect. For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

The key number for server applications is latency — it doesn't matter if you can spawn 200,000 threads if it takes you four seconds to serve a request. And if a thread can't serve a request before its time slice is up, it gets sent to the back of the line and its cache entries get evicted by other threads.

Coroutines have three awesome advantages here:

Really fast switches (it's basically a function call — no need to trap into the kernel)

Scheduling in response to IO events (a coroutine can yield when making a "blocking" IO call and resume when it completes). You can write asynchronous code that looks synchronous!

Better cache behavior (hugely important on modern processors). You can allocate all your coroutine structures together and don't have to suffer all the cache evictions of a context switch every time a coroutine yields.

It would be interesting to see this benchmark repeated with a latency requirement.

Re: Asynchronous IO in Rust

#92

Post by acconsta »

pcwalton wrote: post #69
comex wrote: post #67

Earlier quoted context omitted.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

> It's not going to give you the absolute maximum performance (meaning it's not appropriate for a decent class of program - then again, I suspect Go isn't either), but it's not terrible either. Yeah, this matches our results when we did similar tests. It's definitely faster to use green threads if you're just spawning and shutting down the threads, but if you're actually doing I/O work on those threads, the overhead…

>It's not the fastest way to do I/O, but Go's approach isn't either

Though Go's is faster.

>forego the thread management syscalls and the stack, like nginx does.

The problem is nginx uses state machines, which aren't a general solution. Stackless coroutines allow you to write functions that the compiler (or a clever library) transforms into state machines.

Re: Asynchronous IO in Rust

#93

Post by acconsta »

dbaupp wrote: post #64
acconsta wrote: post #57

Earlier quoted context omitted.

> it's basically just some syntactic niceties over existing functionality. OK, cool! Can you point us to the existing functionality, or any example of it being used for stackless coroutines?

Enums. This HN submission is an example.

State machines aren't a general solution — you have to write a state machine by hand for every application.

Stackless coroutines are — the compiler (or a clever library) transforms your ordinary function into a state machine.

Re: Asynchronous IO in Rust

#94

Post by comex »

Scramblejams wrote: post #85
theseoafs wrote: post #78

Earlier quoted context omitted.

> To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite quickly running out of VM mappings. Okay, so the test you did doesn't actually reflect the use case in practice. Can I expect to reach 200,000 threads if the threads are not all created at exactly the same moment? What if (God forbid) they're doing memory allocation? And if it does work out, will everything be handled effic…

Hope comex replies to your question. Typical green thread usage is spawn-em-as-you-need-em, so if in order to spawn lots of 1:1 threads I need to do it all up front, that could be very limiting or complicating.

Yeah, I just made a mistake - you can increase the maximum number of mappings using /proc/sys/vm/max_map_count; I tried doing that and switching back to normal stack allocation (but still specifying the minimum size of 16KB using pthread_attr_setstacksize) and it doesn't change the number of threads I was able to create.

...in fact, neither did removing the setstacksize call and having default 8MB stacks. I guess this makes sense: of course the extra VM space reserved for the stacks doesn't require actual RAM to back it; there is some page table overhead, but I guess it's not enough to make a significant difference at this number of allocations. Of course, on 32-bit architectures this would quickly exhaust the address space.

If increasing max_map_count hadn't worked, it would still be possible to allocate stacks on the fly - but you would get a bunch of them in one mmap() call, and therefore in one VM mapping, and dole them out in userland. However, in this case guard pages wouldn't separate different threads' stacks, you would have to generate code that manually checks the stack pointer to avoid security issues from stack overflows, rather than relying on crashing by hitting the guard page. Rust actually already does this, mostly unnecessarily; I'm not sure what Go is doing these days but I think it does too. Anyway, given that the above result I suspect this won't be an actual issue, at least until the number of threads goes up by an order of magnitude or something.

Re: Asynchronous IO in Rust

#95

Post by Scramblejams »

comex wrote: post #94
Scramblejams wrote: post #85

Earlier quoted context omitted.

Hope comex replies to your question. Typical green thread usage is spawn-em-as-you-need-em, so if in order to spawn lots of 1:1 threads I need to do it all up front, that could be very limiting or complicating.

Yeah, I just made a mistake - you can increase the maximum number of mappings using /proc/sys/vm/max_map_count; I tried doing that and switching back to normal stack allocation (but still specifying the minimum size of 16KB using pthread_attr_setstacksize) and it doesn't change the number of threads I was able to create. ...in fact, neither did removing the setstacksize call and having default 8MB stacks. I guess thi…

Thanks for your reply. Wonder how other platforms fare.

Re: Asynchronous IO in Rust

#96

Post by comex »

acconsta wrote: post #91
comex wrote: post #67

Earlier quoted context omitted.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

The key number for server applications is latency — it doesn't matter if you can spawn 200,000 threads if it takes you four seconds to serve a request. And if a thread can't serve a request before its time slice is up, it gets sent to the back of the line and its cache entries get evicted by other threads. Coroutines have three awesome advantages here: Really fast switches (it's basically a function call — no need to…

> Scheduling in response to IO events (a coroutine can yield when making a "blocking" IO call and resume when it completes). You can write asynchronous code that looks synchronous!

You can also write "actual" synchronous code which does the same thing in the kernel. :P And there are several CPU schedulers to choose from to fine-tune when things get woken up.

I agree it would be interesting to test latency; actually, the numbers would be much more useful than my four seconds. After all, if thread creation throughput or latency becomes an issue, you can keep a thread pool around, while still retaining the advantages of one connection per thread.

Re: Asynchronous IO in Rust

#97

Post by dbaupp »

acconsta wrote: post #93
dbaupp wrote: post #64

Earlier quoted context omitted.

Enums. This HN submission is an example.

State machines aren't a general solution — you have to write a state machine by hand for every application. Stackless coroutines are — the compiler (or a clever library) transforms your ordinary function into a state machine.

Yes, I know. That's exactly what I mean by syntactic sugar: it is an automated way to write something that could already be written, but is annoying to do by hand.

Re: Asynchronous IO in Rust

#98

Post by dbaupp »

acconsta wrote: post #87
dbaupp wrote: post #63

Earlier quoted context omitted.

1. Yes, as I just said, that's essentially libcore. It doesn't literally offer working dynamic allocations, but I don't see how a cross-platform and cross-use-case freestanding library can do that, too many different environments/constraints to encode an built-in allocator. In fact, core says nothing about how allocation has to work or even if it needs to exist. The rustc distribution has the additional `alloc` and `…

>(Do you have an example of a standards complaint C++ freestanding standard library?) Yes, you've probably heard of libstdc++: https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.what_i... In other words, freestanding C++ allows you to to use new and delete normally (after providing malloc and free). Note that this a minimum requirement . There's nothing stopping you from using STL containers in embedded systems, a…

As you say, the libsupc++ requires you to provide malloc and free, just like libcore.

> probably much of the safety

Only if you want to make life hard for yourself. The standard library is not at all special in its ability to create safe abstractions for `unsafe` code, and doing this makes things so much smoother: you don't have to scatter `unsafe` all over your code, and you let the compiler help you as much as possible. Also, I think there's a lot of nice stuff in `core` too; even string formatting works.

You can still implement equally safe version of them tuned for your environment or, more likely (as the ecosystem develops), use a crate that someone else has written that does it for you. Here's the start of an implementation of a version of `Box` that returns `Result` instead of crashing on allocation failure:

  extern crate core;

  use core::{mem, ops, ptr};

  extern {
      fn malloc(size: usize) -> *const ();
      fn free(p: *const ());
  }

  pub struct MyBox {
      p: *const T,
  }

  impl MyBox {
      fn new(x: T) -> Result, ()> {
          unsafe {
              let p = malloc(mem::size_of::()) as *const T;
              if p.is_null() {
                  Err(())
              } else {
                  ptr::write(p as *mut T, x);
                  Ok(MyBox { p: p })
              }
          }
      }
  }

  impl Drop for MyBox {
      fn drop(&mut self) {
          unsafe {
              drop(ptr::read(self.p));
              free(self.p as *const _);
          }
      }
  }

  // allow `*` to work
  impl ops::Deref for MyBox {
      type Target = T;
      fn deref(&self) -> &T { unsafe { &*self.p } }
  }
This is a safe interface, and is a simplified version of how the built-in box is defined in `std` (well, `alloc`). I've completed the rest of the core functionality `Box` offers, including a demonstration at the bottom: https://play.rust-lang.org/?gist=77f2b15bcc4273846140&versio...

The whole of Rust's standard library (including `Box`, all the containers, all of IO) is built via this process: wrap the raw `unsafe` functionality into safe interfaces that manage things like checking `malloc`'s return value, bounds checking buffer accesses.

> If you want a container that takes user-defined allocators, I'm not even sure what'd you do.

Do exactly what C++ does: have some interface (a trait, in Rust) that an allocator needs to satisfy and have the collections take a type parameter that implements that interface. We "already" know what this looks like in Rust (broadly speaking):

  struct Box {
      p: *const T,
      allocator: A,
  }
The hardest part and sticking point for including this in `std`/`core` is working out a good interface, because it will live forever, needs to cover as much of the problem space as possible and have accommodation for some possible future expansions to the language/library. This isn't such a problem for a container library outside `std`, which is much more flexible because it can be versioned independently, at whatever pace is necessary.

An out-of-tree version aimed at a single use-case (e.g. embedded development) is likely even easier, since the problem space is smaller.

Re: Asynchronous IO in Rust

#99

Post by acconsta »

comex wrote: post #96
acconsta wrote: post #91

Earlier quoted context omitted.

The key number for server applications is latency — it doesn't matter if you can spawn 200,000 threads if it takes you four seconds to serve a request. And if a thread can't serve a request before its time slice is up, it gets sent to the back of the line and its cache entries get evicted by other threads. Coroutines have three awesome advantages here: Really fast switches (it's basically a function call — no need to…

> Scheduling in response to IO events (a coroutine can yield when making a "blocking" IO call and resume when it completes). You can write asynchronous code that looks synchronous! You can also write "actual" synchronous code which does the same thing in the kernel. :P And there are several CPU schedulers to choose from to fine-tune when things get woken up. I agree it would be interesting to test latency; actually,…

>You can also write "actual" synchronous code which does the same thing in the kernel. :P And there are several CPU schedulers to choose from to fine-tune when things get woken up.

You can. But no matter how good your kernel scheduler is, that trip through it is going to cost you. Why? Because it's going to wreck your cache and TLB entries. I think the TLB is a big part of the story, given it's small, specific to the process, and particularly expensive to miss.

http://www.cs.cmu.edu/~chensm/Big_Data_reading_group/papers/...

Anyway, I'm sure you know this already. :) Out of curiosity, did you get a chance to take CS169 before you left Brown?

And I think the kernel already does pool/recycle stacks, but I guess not 200,000 of them given your results. An explicit thread pool would certainly work too.

Re: Asynchronous IO in Rust

#100

Post by acconsta »

dbaupp wrote: post #97
acconsta wrote: post #93

Earlier quoted context omitted.

State machines aren't a general solution — you have to write a state machine by hand for every application. Stackless coroutines are — the compiler (or a clever library) transforms your ordinary function into a state machine.

Yes, I know. That's exactly what I mean by syntactic sugar: it is an automated way to write something that could already be written, but is annoying to do by hand.

Oh, OK. When I hear syntactic sugar, I think of like a macro. Thanks though, I'm glad this is on the core team's radar.
Post reply on HN