Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rust standard library no longer has any scheduling baked into it

mail.mozilla.org

Post reply on HN
Print view View original

41–50 of 71 posts

Re: The Rust standard library no longer has any scheduling baked into it

#41

Post by justincormack »

panzi wrote: post #12
mtanski wrote: post #4

I've also have come full circle on N:M / green threads. Now I'm back to thinking they aren't worth the effort / pain. My experience comes from C / C++ land and not Go and Rust but I think same lessons apply. In experience I ended up using a few different framework from libcoro to Mordor to raw swapcontext(). I was initially sold on the N:M model as a means of having event driven programming without the callback hell.…

I read somewhere that Solaris has the best support for non-blocking disk IO. Does anyone know if that's true/know more about that?

It has certainly had kernel support for a long time, but no evidence that it is significantly better or worse. If Oracle uses it then it is probably good. Almost no one uses the Linux aio support.

Re: The Rust standard library no longer has any scheduling baked into it

#42

Post by pron »

dschatz wrote: post #9
mtanski wrote: post #4

I've also have come full circle on N:M / green threads. Now I'm back to thinking they aren't worth the effort / pain. My experience comes from C / C++ land and not Go and Rust but I think same lessons apply. In experience I ended up using a few different framework from libcoro to Mordor to raw swapcontext(). I was initially sold on the N:M model as a means of having event driven programming without the callback hell.…

A lot of the issues surrounding M:N threads are a result of poor operating system support. The central problem is that a syscall blocks a kernel thread even when there are more user level threads to run. If operating systems supported something like scheduler activations (a 20 year old technique), then this becomes less of a problem. The gist of it is that every time the kernel thread would block, or gets scheduled,…

Right. See the links I posted to Google's proposed Linux kernel changes. I can't wait to use that in Quasar.

Re: The Rust standard library no longer has any scheduling baked into it

#43

Post by justincormack »

qznc wrote: post #37
haberman wrote: post #33

Earlier quoted context omitted.

> Even C has a runtime, whose assumptions can be violated quite thoroughly by other languages. C has a little wrapper around main(), and it has a standard library, but it has no runtime to speak of.

The C runtime consists of stuff like malloc and free. Newer versions of C also require support for thread-local storage for example. It is not a lot, but there definitly is a small runtime called libc. Of course, the C standard also defines the freestanding variant, but there is not even main() anymore.

Its barely a runtime though. I prefer to view Unix as the runtime for C...

Re: The Rust standard library no longer has any scheduling baked into it

#44

Post by dschatz »

qznc wrote: post #35
dschatz wrote: post #9

Earlier quoted context omitted.

A lot of the issues surrounding M:N threads are a result of poor operating system support. The central problem is that a syscall blocks a kernel thread even when there are more user level threads to run. If operating systems supported something like scheduler activations (a 20 year old technique), then this becomes less of a problem. The gist of it is that every time the kernel thread would block, or gets scheduled,…

Why is syscall blocking the central problem? More precisely, what kind of problem is it?

Because if I use M:N threading then logically one of my user threads blocked and a different one should run during my timeslice. The kernel however is unaware of how I use the kernel thread and will block, believing that I cannot proceed.

1:1 threading lacks this problem but operations such as creating threads or deleting them require syscalls and are therefore relatively expensive.

Re: The Rust standard library no longer has any scheduling baked into it

#45

Post by pyotrgalois »

jerf wrote: post #11
mtanski wrote: post #7

Earlier quoted context omitted.

This strikes me a pragmatic decision driven by wanting to reduce complexity and be interoperable with 3rd party libraries (mostly C code). If you read my other comment here, I outlined the problems I ran into with the N:M threading model (although mostly in C land). It strikes me as Rust ran into the same issues. I'm beginning to think that you can't make N:M well unless you take the approach that language X itself i…

"I don't know Erlang's stance on this." Erlang's stance is probably irrelevant to anyone in the Go/Rust/C/C++ etc community; it does so many things differently that its experiences are unlikely to be useful. I'm pretty sure it's M:N, but, for instance, it also trivially has a NOT-stop-the-world GC, which is something that not many languages can say. Not only that it isn't "stop the world" but that it's also fairly tr…

Why do you think it pays for these characteristics? I'm interested on your point of view.

Re: The Rust standard library no longer has any scheduling baked into it

#46

Post by Daishiman »

Pacabel wrote: post #23

I really want to be able to use Rust, but then I keep seeing changes that seem to go around and around in circles. I do appreciate that it's very very hard to get things right the first time around and I understand that some experimentation is necessary. But when it comes to Rust it seems like one approach is deemed the correct way, then it's built and used, and like most things it has flaws. Then the complete opposi…

Rust is pre-alpha; it's as green as it can be, and it is first and foremost (for the moment) a PL research project. You should not be using it in production, or expecting things to not break between releases.

That said, breaking changes are becoming less and less common. The language syntax has been mostly stabilized and most changes are going into corner cases and managing modularity, and much of the possible experimentation has been fenced to the post-1.0 release.

Re: The Rust standard library no longer has any scheduling baked into it

#47

Post by exDM69 »

Pacabel wrote: post #23

I really want to be able to use Rust, but then I keep seeing changes that seem to go around and around in circles. I do appreciate that it's very very hard to get things right the first time around and I understand that some experimentation is necessary. But when it comes to Rust it seems like one approach is deemed the correct way, then it's built and used, and like most things it has flaws. Then the complete opposi…

I see the changes in Rust as a positive, not a negative thing. I do understand that it may become a burden for a Rust programmer and is a major hindrance if you'd actually want to go to production with it. But even this fairly major change should not break a lot of existing Rust code.

But this allows Rust to do bleeding edge research instead of finalizing on a strict specification before knowing if the design will work in practice. It is also very refreshing to see programmers admit that their choice was wrong and not insisting on staying with a bad choice that was made earlier.

Consider Rust to be a practical research project, at least for now.

Re: The Rust standard library no longer has any scheduling baked into it

#48

Post by tokenrove »

haberman wrote: post #33
jerf wrote: post #5

Earlier quoted context omitted.

On the flip side, though, those assumptions are also what lets a runtime actually do things for you. In the limit, a runtime that assumes nothing is simply machine language. Even C has a runtime, whose assumptions can be violated quite thoroughly by other languages. The less the language actually specifies, the more likely you are to encounter the situation where two Rust libraries can't work together because they ac…

> Even C has a runtime, whose assumptions can be violated quite thoroughly by other languages. C has a little wrapper around main(), and it has a standard library, but it has no runtime to speak of.

That little wrapper is a runtime (after all, what does crt stand for?); depending on the platform, it's certainly not so negligible that you can say it doesn't exist.

Re: The Rust standard library no longer has any scheduling baked into it

#49

Post by mtanski »

justincormack wrote: post #41
panzi wrote: post #12

Earlier quoted context omitted.

I read somewhere that Solaris has the best support for non-blocking disk IO. Does anyone know if that's true/know more about that?

It has certainly had kernel support for a long time, but no evidence that it is significantly better or worse. If Oracle uses it then it is probably good. Almost no one uses the Linux aio support.

Oracle uses AIO on most operating systems. It implements it's own block and buffer cache so the kernel page cache just gets in the way. Since it does it's own block cache, the block size and alignment limitations are not really a problem for it. KVM also uses AIO for similar reasons (guess OS has it's own page cache so we want to avoid double caching).

AIO on Linux was basically designed for Oracle. Only a few classes of apps can really benefit from it. It also doesn't really guarantee to be non-blocking as many filesystem metadata operations do block (read the long threads on Linux mailing lists).

Unless you're a special class of app that can live with the AIO limitations then you have to stick to blocking IO. Otherwise you're stuck rewriting the page cache and it's unlikely that you'll do a better job the the built in OS.

I think Solaris has similar limitations compared to Linux. Windows has a better API with its async overlapped file API and even there various NTFS metadata operations can block.

Re: The Rust standard library no longer has any scheduling baked into it

#50

Post by mtanski »

pron wrote: post #40
justincormack wrote: post #38

Earlier quoted context omitted.

Have you got a reference for user-scheduled OS threads? Hadn't come across that...

Video: https://www.youtube.com/watch?v=KXuZi9aeGTw Slides: http://www.linuxplumbersconf.org/2013/ocw//system/presentati...

Thanks for providing the link and it does seam like an interesting proposal.

To summarize:

Switching into ring0 (kernel is not that expensive)

We're going to stay with the 1:1 thread model

We're going to provide an new syscall to hint to the scheduler which thread to switch to.

Provided we have time slice still left the scheduler can do that almost instantly since picking the task to run next is expensive (their data).

This isn't scheduler activations but a yieldTo().

This method avoids all sorts of problems with 3rd party libraries namely avoiding TLS problems.

This doesn't do anything about block IO.

Post reply on HN