Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Asynchronous IO in Rust

medium.com

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Asynchronous IO in Rust

#81

Post by Jweb_Guru »

theseoafs wrote: post #79
Jweb_Guru wrote: post #76

Earlier quoted context omitted.

The point is: who cares? Do you really need 3 million threads (particularly, 3 million threads that aren't as efficient as ordinary threads for many tasks)? Ostensibly Google does about 3 billion queries a day; that's less than 35,000 queries per second. Obviously, there are many services underlying any Google search, but most of them will be distributed among multiple cores, machines, and datacenters. So what eventu…

The point isn't that I need 3 million threads. The point is that I can easily write code that is correct and behaves predictably even if I end up throwing some arbitrary number of threads at the processor. As I view them, green threads are a useful abstraction just like objects or functions. Do you need something computed, not necessarily now, but at some point? Just make a green thread to compute it and check on the…

> The point is that I can easily write code that is correct and behaves predictably even if I end up throwing some arbitrary number of threads at the processor.

But you can't. Green threads have scalability limitations too, and their performance is not predictable (except possibly by the creators of the runtime) in any language or on any processor that does automatic preemption. That's like saying you use i64 instead of i32 because now you can do calculations without worrying about overflow. It may be true for some classes of problem, but it's hardly a different programming paradigm like you're making it out to be. They're definitely not a different abstraction from threads (which are themselves an abstraction!).

Re: Asynchronous IO in Rust

#82

Post by Jweb_Guru »

dan00 wrote: post #68
pcwalton wrote: post #58

Earlier quoted context omitted.

> For example, in a native threading model, it is an awful idea to spawn a thread for every incoming connection on a server. It's the easy way to write it, but it's wrong. With a green threading model, though, that's easy and efficient. A thread per connection isn't wrong, though. You're begging the question by assuming that green threads are faster than native threads. I'm specifically arguing against that.

I always thought that the main arguments for green threads was their cheaper context switch compared to native threads and their less usage of memory. The next point might be if it's harder to get the same performance characteristics with native threads on different platforms.

Yeah, outside of Linux threads are dog slow. But Linux's implementation demonstrates that they don't have to be, and if you have serious scalability problems you're going to know your architecture well in advance.

Re: Asynchronous IO in Rust

#83

Post by acconsta »

steveklabnik wrote: post #72
acconsta wrote: post #49

Earlier quoted context omitted.

OOM is an abort, not a panic (contrary to the official docs, interestingly) > you're free to provide an alternative stdlib But like... should you have to? > for those rare occasions In kernel programming, allocation failures are common and handling them is essential. To quote my (very talented) classmate, who actually wrote part of a kernel in Rust: "The only really major issue is with how allocation failure works, w…

> OOM is an abort, not a panic (contrary to the official docs, interestingly) Where in the docs do we say this? I'd like to fix it.

The docs (which are generally very good, especially for a language this young) should:

1. Distinguish between abort and panic.

2. Explain that allocation failure is an abort.

3. Explain that a vec can allocate up to twice its nominal size, which is a common cause of OOMs. To their credit, the docs do explain that vec over-allocates, but not that it allocates double the memory. They could also explain that you don't get that memory back unless you ask for it, even if you pop() or remove() elements.

Scenario:

I'm a new systems programmer excited to try Rust (fast and safe? wow!). I have 8GB of memory in my computer. I push ~4GB of data into a vector so I can process it, something I'm used to doing in Python.

POW! My program exits with no error message. What happened? I look at the error handling docs:

https://doc.rust-lang.org/book/error-handling.html

Well, it sure wasn't a recoverable failure! Must have been a panic. So I look at the docs for vec.push:

https://doc.rust-lang.org/std/vec/struct.Vec.html

"Panics if the number of elements in the vector overflows a usize."

A usize is big, right? I didn't overflow that. I guess I could have run out of memory, but I have 8GB. That should be plenty!

Re: Asynchronous IO in Rust

#84

Post by sillysaurus3 »

wtetzner wrote: post #77
sillysaurus3 wrote: post #32

Earlier quoted context omitted.

The interface is different because if you use a green thread within a GC'd language, you can do anything -- change anything, interact with anything -- and nothing will crash. No locking involved, and no hidden locking under the hood. It's not meant to be a performance boost, but a way of writing code where "might this crash?" is a question you're never bothered to ask. It's wonderfully freeing. Not true of native thr…

> a way of writing code where "might this crash?" is a question you're never bothered to ask. Is that actually true? Yes, you won't get a partially written int, but you could still run into consistency issues if you don't lock. For example, if you need thread A to update both x and y to have a consistent state, and B is reading both x and y, A might update x, then the scheduler would switch to B, and be might read th…

It will do what you just described. And what you described won't crash.

If you use the int as an offset into raw memory, then sure, that will. But if you use it as an offset into a list, then it won't. At most it will cause an exception to be generated. That will cause the reader thread to die, because it was the code that caused the exception, but it still won't crash.

Re: Asynchronous IO in Rust

#85

Post by Scramblejams »

theseoafs wrote: post #78
comex wrote: post #67

Earlier quoted context omitted.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

> To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite quickly running out of VM mappings. Okay, so the test you did doesn't actually reflect the use case in practice. Can I expect to reach 200,000 threads if the threads are not all created at exactly the same moment? What if (God forbid) they're doing memory allocation? And if it does work out, will everything be handled effic…

Hope comex replies to your question. Typical green thread usage is spawn-em-as-you-need-em, so if in order to spawn lots of 1:1 threads I need to do it all up front, that could be very limiting or complicating.

Re: Asynchronous IO in Rust

#86

Post by acconsta »

Manishearth wrote: post #61
acconsta wrote: post #54

Earlier quoted context omitted.

1. In practice most embedded toolchains will give you some of the standard library. A standards compliant C++ freestanding library provides new and delete, for example. 2. The libcore allocator API is marked unstable, so even if you go through the trouble of implementing it, how long will it last?

> will give you some of the standard library. That's what libcore is. A lot of the stdlib is reexports over libcore. new and delete are C++ isms, their Rust counterparts are `Box::new` (and destructors are automatic). `Box::new`, like `new`, has the OOM issue. If you're okay with that, you are free to link to the `alloc` crate, which gives you `Box` without pulling in additional deps. There are plans for in-place box…

Just to be clear, the C++ equivalent of Box is unique_ptr. I don't see equivalents of new and delete, but I might be missing them.

I see Box::from_raw and core::ops::Placer. Is that what you mean by in-place boxing? But now there are two problems:

You have to manually allocate a buffer of the right size, check if the pointer is null, and box it. For every allocation. Forget a null check? Allocate too small a buffer? Guess what, you've got undefined behavior!

It also doesn't generalize to other parts of the standard library. Did you want to use Rust's built in containers in your kernel? Well, sorry, you're going to have to write your own ones that don't panic on allocation failure.

Re: Asynchronous IO in Rust

#87

Post by acconsta »

dbaupp wrote: post #63
acconsta wrote: post #54

Earlier quoted context omitted.

1. In practice most embedded toolchains will give you some of the standard library. A standards compliant C++ freestanding library provides new and delete, for example. 2. The libcore allocator API is marked unstable, so even if you go through the trouble of implementing it, how long will it last?

1. Yes, as I just said, that's essentially libcore. It doesn't literally offer working dynamic allocations, but I don't see how a cross-platform and cross-use-case freestanding library can do that, too many different environments/constraints to encode an built-in allocator. In fact, core says nothing about how allocation has to work or even if it needs to exist. The rustc distribution has the additional `alloc` and `…

>(Do you have an example of a standards complaint C++ freestanding standard library?)

Yes, you've probably heard of libstdc++:

https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.what_i...

In other words, freestanding C++ allows you to to use new and delete normally (after providing malloc and free). Note that this a minimum requirement. There's nothing stopping you from using STL containers in embedded systems, and people do (especially now that STL containers can take user defined allocators).

It seems like Rust is trying to achieve the same result with libcore. That's good! Thanks for clarifying.

But now you lose all the nice parts of the standard library, and probably much of the safety. If you want to use Box, you have to manually do the allocation, check the pointer, and construct the box. Buffer too small? Forget the pointer check? Welcome to undefined behavior.

If you want a container, you have to roll your own. If you want a container that takes user-defined allocators, I'm not even sure what'd you do.

Re: Asynchronous IO in Rust

#88

Post by trentnelson »

mtanski wrote: post #74
justincormack wrote: post #70

Earlier quoted context omitted.

Well, apparently not on Windows. But this is a kernel interface issue not a language issue, and clearly needs fixing now we have SSDs where having thousands of outstanding requests is useful, if not required for performance, unless the hardware APIs are going to change (maybe if they get memory interfaces this does change).

The async story on windows is better, but not great. In many cases the async calls on windows silently block due to a number of special cases (that are not so special).

(Presuming you're referring to this: https://support.microsoft.com/en-us/kb/156932)

Those are all entirely reasonable things to block for if you understand the mechanics behind why those calls may block. And none of it should be fast-path stuff; it's easy to architect a solution where you avoid those things except in corner cases.

The other thing you're missing is that when you architect around Windows completion ports and threadpool I/O facilities, it doesn't matter if one thread blocks every so often. Windows will detect this and schedule another one to run, such that there is one active thread scheduled on every core.

I/O completion ports facilitate thread-agnostic I/O completion; the thread that blocked because it was extending an encrypted file won't impede the latency of other clients because there is no thread-specific association.

I exploit all of this with PyParallel. http://pyparallel.org

For more details on asynchronous I/O on Windows and why it's fundamentally better than the UNIX I/O model in every possible way: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...

Re: Asynchronous IO in Rust

#89

Post by trentnelson »

justincormack wrote: post #70
mtanski wrote: post #40

I think a lot of folks think of Network IO when they say Asynchronous IO. That's only half the story, unless you're just building proxies and caches you have to deal with Disk IO at some point in time. And, async disk IO is horrible in every OS / language.

Well, apparently not on Windows. But this is a kernel interface issue not a language issue, and clearly needs fixing now we have SSDs where having thousands of outstanding requests is useful, if not required for performance, unless the hardware APIs are going to change (maybe if they get memory interfaces this does change).

> Well, apparently not on Windows.

Agreed! http://pyparallel.org

> But this is a kernel interface issue not a language issue,

Absolutely agree! The thing that surprises me most about Rust and Go in particular is that they're trying to solve an OS problem with new language abstractions. You don't need a new language -- what you really want is thread-agnostic I/O primitives, thread-agnostic "threadpool" primitives, and completion-oriented APIs that provide both synchronous and asynchronous primitives. Basically, you want what Windows has.

Re: Asynchronous IO in Rust

#90

Post by acconsta »

theseoafs wrote: post #78
comex wrote: post #67

Earlier quoted context omitted.

I was curious what the actual state was of the "modern Linux kernel" pcwalton mentioned, so I tried running a test program to create a million threads on a VM - x86-64 with 8GB of RAM, Linux 4.0. For comparison, Go apparently uses about 4KB per goroutine, so it should be possible to create somewhat under 2 million goroutines. To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite…

> To be fair, I allocated the stacks manually in one large allocation; otherwise it dies quite quickly running out of VM mappings. Okay, so the test you did doesn't actually reflect the use case in practice. Can I expect to reach 200,000 threads if the threads are not all created at exactly the same moment? What if (God forbid) they're doing memory allocation? And if it does work out, will everything be handled effic…

That doesn't seem that unrealistic — you could allocate your stacks using slab allocation, for example. I wonder why the Kernel allocator doesn't do a better job though.
Post reply on HN