Asynchronous IO in Rust • Hacker News Board

Re: Asynchronous IO in Rust

#61

Post by Manishearth » Thu, Sep 17, 2015, 7:28 AM UTC

acconsta wrote: post #54

dbaupp wrote: post #51

Earlier quoted context omitted.

> But like... should you have to? This is the status quo (in all languages) if you're doing embedded work: you lose most of the standard library. Rust goes slightly beyond that default by bundling `core`, which is designed for these environments and serves as a building block on which embedded/bare-metal development can flourish. (Cargo works fine with such things, so one can even distribute alternate standard librar…

1. In practice most embedded toolchains will give you some of the standard library. A standards compliant C++ freestanding library provides new and delete, for example. 2. The libcore allocator API is marked unstable, so even if you go through the trouble of implementing it, how long will it last?

> will give you some of the standard library.

That's what libcore is. A lot of the stdlib is reexports over libcore.

new and delete are C++ isms, their Rust counterparts are `Box::new` (and destructors are automatic). `Box::new`, like `new`, has the OOM issue. If you're okay with that, you are free to link to the `alloc` crate, which gives you `Box` without pulling in additional deps.

There are plans for in-place boxing, and AFAICT that API will be available to embedded users.

Re: Asynchronous IO in Rust

#62

Post by kentonv » Thu, Sep 17, 2015, 7:30 AM UTC

saaadhu wrote: post #46

mtanski wrote: post #40

I think a lot of folks think of Network IO when they say Asynchronous IO. That's only half the story, unless you're just building proxies and caches you have to deal with Disk IO at some point in time. And, async disk IO is horrible in every OS / language.

Care to explain why async disk IO is horrible? AFAIK, the kernel APIs only deal with abstract handles and descriptors - they aren't written specifically for disk IO or network IO. And C#'s async/await and BeginXXX also work fine with all kinds of IO.

On Unix, the APIs for async disk I/O are completely different from the APIs for async network I/O. This is because, on Unix, the APIs for network I/O are "readiness"-based (they tell you when data is available in the read buffer or space is available in the write buffer), but a disk file handle is always ready (except at EOF). So, for disk I/O you need a "completion" model, where you queue an operation and then get a callback when it is done. This is a very different kind of API, and as a result it does not fit well with the network I/O APIs, but of course if you are doing async you probably need to do both disk and network at the same time, so you need the APIs to play nice with each other (you need a single event loop that waits for both kinds of events).

(On Windows, as I understand it, you can do completion-based I/O on all handle types.)

And then, even if you manage to use async disk I/O, you can only really use it for reads and writes. Filesystem calls that manipulate the directory tree usually don't have async versions. At some point you have to give up and do things in threads.

Re: Asynchronous IO in Rust

#63

Post by dbaupp » Thu, Sep 17, 2015, 7:38 AM UTC

acconsta wrote: post #54

dbaupp wrote: post #51

Earlier quoted context omitted.

> But like... should you have to? This is the status quo (in all languages) if you're doing embedded work: you lose most of the standard library. Rust goes slightly beyond that default by bundling `core`, which is designed for these environments and serves as a building block on which embedded/bare-metal development can flourish. (Cargo works fine with such things, so one can even distribute alternate standard librar…

1. In practice most embedded toolchains will give you some of the standard library. A standards compliant C++ freestanding library provides new and delete, for example. 2. The libcore allocator API is marked unstable, so even if you go through the trouble of implementing it, how long will it last?

1. Yes, as I just said, that's essentially libcore.

It doesn't literally offer working dynamic allocations, but I don't see how a cross-platform and cross-use-case freestanding library can do that, too many different environments/constraints to encode an built-in allocator. In fact, core says nothing about how allocation has to work or even if it needs to exist.

The rustc distribution has the additional `alloc` and `collections` crates layered on top of `core`, which offers some extra functionality when allocation does exist (without assuming other OS support, like IO), and the allocator used is pluggable.

(Do you have an example of a standards complaint C++ freestanding standard library?)

2. There will be some way to do this long term. I can offer no guarantees that it will be what's there right now, but something will exist. This is also something that has been on the radar for a long time, but isn't core to the language. In any case, what is in core is essentially just optimising stack usage, all of the sematics it offers can be done with normal function calls, e.g. `Box::new` is literally just a normal function and perfectly implementable in any environment, if you have some allocation routine (the compiler/language doesn't need to know about either). So the unstable libcore functionality is important and desirable, but not killer.

There's also the meaning of "allocators" as in allowing `Vec` (etc.) to use some other algorithm than what Rust does by default; again something desirable and been on the radar for a while, but not key to the language. I don't think this needs any new language features, just people to explore the library space. (NB. this is different to what libcore offers now, which is basically what is called "placement new" in C++.)

Re: Asynchronous IO in Rust

#64

Post by dbaupp » Thu, Sep 17, 2015, 7:38 AM UTC

acconsta wrote: post #57

dbaupp wrote: post #52

Earlier quoted context omitted.

The first point has been on the radar for quite a while (i.e. years), but it's really non-trivial, and not at all necessary for the core language (hence not a priority for stability) since it's basically just some syntactic niceties over existing functionality.

> it's basically just some syntactic niceties over existing functionality. OK, cool! Can you point us to the existing functionality, or any example of it being used for stackless coroutines?

Enums. This HN submission is an example.

Re: Asynchronous IO in Rust

#65

Post by dbaupp » Thu, Sep 17, 2015, 7:39 AM UTC

Manishearth wrote: post #61

acconsta wrote: post #54

Earlier quoted context omitted.

1. In practice most embedded toolchains will give you some of the standard library. A standards compliant C++ freestanding library provides new and delete, for example. 2. The libcore allocator API is marked unstable, so even if you go through the trouble of implementing it, how long will it last?

> will give you some of the standard library. That's what libcore is. A lot of the stdlib is reexports over libcore. new and delete are C++ isms, their Rust counterparts are `Box::new` (and destructors are automatic). `Box::new`, like `new`, has the OOM issue. If you're okay with that, you are free to link to the `alloc` crate, which gives you `Box` without pulling in additional deps. There are plans for in-place box…

> There are plans for in-place boxing, and AFAICT that API will be available to embedded users.

Already exists, unstably: http://doc.rust-lang.org/nightly/core/ops/trait.Placer.html

Re: Asynchronous IO in Rust

#66

Post by jabl » Thu, Sep 17, 2015, 7:47 AM UTC

cwp wrote: post #36

jerf wrote: post #3

If you use threads, green or otherwise, you don't have to "implement" special code for composing things together, you get the full set of tools for composing code together, which includes, in passing, state machines, among all the other things it includes. This basically implements an Inner Platform Effect of an internal data-based language for concurrency that the language interprets, which will A: forever be weaker…

Isn't this a rehash of the C10K problem[1] from a decade ago? That was pretty much resolved in favour of single-threading and asynchronous IO, with Nginx and Node.js replacing Apache and Ruby as the platforms that the cool kids use. So, if threads are the way to go today, what has changed in the last 10 years to turn the conventional wisdom on it's head? 64-bit processors and servers with more memory? Hypervisors/con…

Disclaimer: I'm not an expert in this area, merely an interested bystander.

So, AFAIU the C10K approach is still the correct approach. Although perhaps with current hw and Linux being able to decently handle quite a lot of threads since NPTL, one should nowadays call it the C100K problem?

What has changed is a change in focus. Few people do entirely static sites anymore, and for static assets (images etc.) everybody uses nginx or other nonblocking approaches anyway. Perhaps also nginx/etc. as a reverse proxy. So there is no argument that nonblocking approaches are better for handling a lot of potentially slow connections.

But for the "core" functionality of a network application, that people are actually spending times programming rather than using an off the shelf solution like nginx, nonblocking vs. threads doesn't matter that much, there's all kinds of DB calls, CPU intensive work to do, etc. So people are (correctly) asking whether we can create nonblocking code but with an easier to use programming model (to the extent it does matter for performance) or whether to just use threads.

Re: Asynchronous IO in Rust

#67

Post by comex » Thu, Sep 17, 2015, 7:52 AM UTC

theseoafs wrote: post #33

pcwalton wrote: post #31

Earlier quoted context omitted.

I'm confused. To me, green threads are just an implementation detail. The fact that green threading is being used shouldn't leak into the interface. To take Go for an example, you could perfectly well write a conforming implementation of Go that used 1:1 native threading: it would just have different performance characteristics and things like LockOSThread() would become a no-op. Could you elaborate on what you consi…

This is backwards. Green threading can be an implementation detail, but in languages where it's a key feature, green threads let you write code that would otherwise be incorrect. For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite quickly running out of VM mappings. I set the stack size to the pthread minimum of 16KB (actually, I tried cheating and making it smaller, but it crashed, so I gave up - not a good idea anyway). The threads waited for an indication from the main thread, sent after thread creation was done, to exit; in an attempt to avoid the overhead associated with pthread conditions, I just used futex directly:

    while (!ready)
        assert(!syscall(SYS_futex, &ready, FUTEX_WAIT, 0, NULL, NULL, 0));

The program caused the kernel to hit OOM (rather ungracefully!) somewhere around 270,000 threads. To see how long it took while ensuring all the threads actually ran, I reduced the thread count to 200,000, had it join all the threads at the end, and timed this whole process: after the first run it took about 4 seconds. (The first run was considerably slower, but that isn't a big deal for a server, which is the most important use case for having such a large number of goroutines/threads.) Therefore, the C version uses about 20 microseconds and 32 KB of memory per thread.

For completeness, I also tested a similar Go program on Go 1.4 (the version available from Debian on the VM); it actually got up to 3,150,000 before OOM, and took 9 seconds to do 2 million - 4.5 microseconds and 2.7KB per thread.

In other words, Linux is about an order of magnitude slower at managing a lrge number of threads. That looks pretty bad, but on the other hand, it's not that much in absolute terms! I'm pretty sure most server programs don't need more than 250,000 simultaneous connections (or can afford to spend more than 8GB of RAM on them) and don't mind spending an extra 20 microseconds to initiate a connection, so if operating systems other than Linux aren't a concern, they could be written to create a thread per connection without too much trouble. It's not going to give you the absolute maximum performance (meaning it's not appropriate for a decent class of program - then again, I suspect Go isn't either), but it's not terrible either.

I'd like to see it improve. I wouldn't be surprised if there is (still) some low hanging fruit; do kernel developers actually care about this use case?

(And yes, I know this doesn't really test performance of the scheduler during sustained operation. That's its own can of worms.)

Re: Asynchronous IO in Rust

#68

Post by dan00 » Thu, Sep 17, 2015, 7:59 AM UTC

pcwalton wrote: post #58

theseoafs wrote: post #33

Earlier quoted context omitted.

This is backwards. Green threading can be an implementation detail, but in languages where it's a key feature, green threads let you write code that would otherwise be incorrect. For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient.

> For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient. A thread per connection isn't wrong, though. You're begging the question by assuming that green threads are faster than native threads. I'm specifically arguing against that.

I always thought that the main arguments for green threads was their cheaper context switch compared to native threads and their less usage of memory.

The next point might be if it's harder to get the same performance characteristics with native threads on different platforms.

Re: Asynchronous IO in Rust

#69

Post by pcwalton » Thu, Sep 17, 2015, 8:30 AM UTC

comex wrote: post #67

theseoafs wrote: post #33

Earlier quoted context omitted.

This is backwards. Green threading can be an implementation detail, but in languages where it's a key feature, green threads let you write code that would otherwise be incorrect. For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

> It's not going to give you the absolute maximum performance (meaning it's not appropriate for a decent class of program - then again, I suspect Go isn't either), but it's not terrible either.

Yeah, this matches our results when we did similar tests. It's definitely faster to use green threads if you're just spawning and shutting down the threads, but if you're actually doing I/O work on those threads, the overhead quickly drops down.

It's not the fastest way to do I/O, but Go's approach isn't either. The fastest way to do I/O is to forego the thread management syscalls and the stack, like nginx does.

Re: Asynchronous IO in Rust

#70

Post by justincormack » Thu, Sep 17, 2015, 9:25 AM UTC

mtanski wrote: post #40

I think a lot of folks think of Network IO when they say Asynchronous IO. That's only half the story, unless you're just building proxies and caches you have to deal with Disk IO at some point in time. And, async disk IO is horrible in every OS / language.

Well, apparently not on Windows. But this is a kernel interface issue not a language issue, and clearly needs fixing now we have SSDs where having thousands of outstanding requests is useful, if not required for performance, unless the hardware APIs are going to change (maybe if they get memory interfaces this does change).