Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rust standard library no longer has any scheduling baked into it

mail.mozilla.org

Post reply on HN
Print view View original

21–30 of 71 posts

Re: The Rust standard library no longer has any scheduling baked into it

#21

Post by EvanMiller »

mtanski wrote: post #4

I've also have come full circle on N:M / green threads. Now I'm back to thinking they aren't worth the effort / pain. My experience comes from C / C++ land and not Go and Rust but I think same lessons apply. In experience I ended up using a few different framework from libcoro to Mordor to raw swapcontext(). I was initially sold on the N:M model as a means of having event driven programming without the callback hell.…

The Erlang VM is a successful implementation of N:M. It addresses some but not all of the issues you mentioned.

* IIRC each new process consumes 284 bytes so it's not "spending lots of memory on creating stacks for your green threads"

* Synchronization is a non-issue as everything is message-passing.

* Disk I/O is blocking, but the VM has dedicated disk threads.

* Dealing with 3rd party code is a problem, which is why relatively few libraries exist for Erlang. However, there is some hope now that NIFs (native interface functions, i.e. C code) can interact with the scheduler, i.e. report how much time has been used and return control if necessary.

You are right that "Building a correct highly concurrent scheduler is no easy task." The Erlang scheduler was not an easy task, and has received a ton of work from extremely talented engineers over the course of decades. Definitely worth a look to see how N:M can be made to work well.

Re: The Rust standard library no longer has any scheduling baked into it

#22

Post by jeltz »

girvo wrote: post #19
jeltz wrote: post #8

Earlier quoted context omitted.

Erlang has been using M:N very successfully and managed to have good latency and fairness while sacrificing performance. The hard part about M:N is doing it without a VM. EDIT: To clarify I believe you are too harsh about M:N, and that Erlang is excellent proof that M:N has its use cases.

Erlang and it's BEAM VM are so different to other mainstream languages, that I'm not sure that proves that much...

It proves that there are cases where M:N scheduling is the right choice. But it does not prove if it is the right choice for language implementation X (where X in this case is Rust).

Re: The Rust standard library no longer has any scheduling baked into it

#23

Post by Pacabel »

I really want to be able to use Rust, but then I keep seeing changes that seem to go around and around in circles. I do appreciate that it's very very hard to get things right the first time around and I understand that some experimentation is necessary. But when it comes to Rust it seems like one approach is deemed the correct way, then it's built and used, and like most things it has flaws. Then the complete opposite approach is tried but it runs into a different set of flaws, usually the ones that caused the other approach to be chosen in the first place. The Rust home page describes it as a "practical language" but with all of these back and forth changes it is really really difficult to actually use it. I don't know if I can keep waiting for it. When I've got new code to write I'm just going to have to use a language that I know I can depend on for more than a month or two, like Go or Haskell or Ruby.

- Pacabel

Re: The Rust standard library no longer has any scheduling baked into it

#24

Post by dbaupp »

Pacabel wrote: post #23

I really want to be able to use Rust, but then I keep seeing changes that seem to go around and around in circles. I do appreciate that it's very very hard to get things right the first time around and I understand that some experimentation is necessary. But when it comes to Rust it seems like one approach is deemed the correct way, then it's built and used, and like most things it has flaws. Then the complete opposi…

Which back and forth changes are you referring to?

(Also, yeah, Rust is pre-release and is definitely not suitable for people who don't like making minor changes regularly if they're tracking master (or less minor changes occasionally if tracking the 0.x releases).)

Re: The Rust standard library no longer has any scheduling baked into it

#25

Post by wtracy »

sanxiyn wrote: post #20
bsdetector wrote: post #6

Earlier quoted context omitted.

Half a year ago: "I wouldn't be surprised to see Rust at least abandon M:N soon though once they start really optimizing performance." With Rust I'm sure they'll drop M:N, if not now then eventually, like essentially everybody else in CS history . It seems great on a superficial level, but when you really start caring about performance and latency and fairness it's the pits. My question is when Go will drop it. Imagi…

Haskell people seem happy with their M:N scheduler. Is something Haskell-specific helping them? https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Schedul...

Pure functions allow the runtime to make an entire set of assumptions that aren't safe with other languages.

Re: The Rust standard library no longer has any scheduling baked into it

#26

Post by thirsteh »

wtracy wrote: post #25
sanxiyn wrote: post #20

Earlier quoted context omitted.

Haskell people seem happy with their M:N scheduler. Is something Haskell-specific helping them? https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Schedul...

Pure functions allow the runtime to make an entire set of assumptions that aren't safe with other languages.

Given that you 'forkIO' functions in the IO monad, i.e. impure functions, what do you mean by that? 'forkIO' in Haskell is essentially the same as dispatching a green thread in any other language, e.g. the same as 'go' in Go.

Haskell/GHC's threads are primarily a concurrency construct, not a parallelism one. You 'forkIO' a bunch of impure functions and communicate between them using channels, STM, etc., exactly the same as in other languages. For parallelism in pure functions there are much better tools--par seq, parallel strategies, repa, Data Parallel Haskell, etc.

Re: The Rust standard library no longer has any scheduling baked into it

#27

Post by thirsteh »

girvo wrote: post #19
jeltz wrote: post #8

Earlier quoted context omitted.

Erlang has been using M:N very successfully and managed to have good latency and fairness while sacrificing performance. The hard part about M:N is doing it without a VM. EDIT: To clarify I believe you are too harsh about M:N, and that Erlang is excellent proof that M:N has its use cases.

Erlang and it's BEAM VM are so different to other mainstream languages, that I'm not sure that proves that much...

Being different from the mainstream isn't a bad thing. For the sake of argument, Rust is very different from the mainstream too.

Re: The Rust standard library no longer has any scheduling baked into it

#28

Post by blt »

mtanski wrote: post #4

I've also have come full circle on N:M / green threads. Now I'm back to thinking they aren't worth the effort / pain. My experience comes from C / C++ land and not Go and Rust but I think same lessons apply. In experience I ended up using a few different framework from libcoro to Mordor to raw swapcontext(). I was initially sold on the N:M model as a means of having event driven programming without the callback hell.…

great post, thanks for sharing your real life experience!

Re: The Rust standard library no longer has any scheduling baked into it

#29

Post by blt »

I think this is a good decision. A language that aims to compete with C/C++ needs to expose the low-level fundamental building blocks that the hardware and OS provide. Anything high level like a mandatory GC or scheduler will scare away systems programmers.

Re: The Rust standard library no longer has any scheduling baked into it

#30

Post by axaxs »

mtanski wrote: post #4

I've also have come full circle on N:M / green threads. Now I'm back to thinking they aren't worth the effort / pain. My experience comes from C / C++ land and not Go and Rust but I think same lessons apply. In experience I ended up using a few different framework from libcoro to Mordor to raw swapcontext(). I was initially sold on the N:M model as a means of having event driven programming without the callback hell.…

Don't count go out, yet. Its green thread method works really well, and it's moving to contiguous stack next release. I've never ran into any problems with this setup as long as I'm smart about control. I can imagine that trying to replicate myself in c would be an absolute nightmare, so I trust these more experienced folks, and they've not let me down yet...
Post reply on HN