Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chinese Whispers in Rust

thornydev.blogspot.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Chinese Whispers in Rust

#21

Post by redbad »

coldtea wrote: post #15
ZoFreX wrote: post #8

Earlier quoted context omitted.

Oh, come on. Not only is your comment needlessly insulting but it's completely unconstructive. What types of CSP things should you do in Rust, then? Which should you not? Why is this code not a demonstration of a slowness in Rust but a 'bad idea'? How would you implement it in a more idiomatic manner in Rust?

> Oh, come on. Not only is your comment needlessly insulting but it's completely unconstructive. I find his comment pretty spot on. If anything it was the "don't do CSP in Rust" comment that was unconstructive and passive aggresively insulting. Instead of understanding what he read and the constrains it shows, the commenter preffered to just piss on the language. As for the example, it was obviously "retarted" or rat…

    > CSP and channels are meant for getting real (cpu 
    > intensive) work done. You don't just use threads (or 
    > greenthreads) "because concurrency".
The whole point of green threads is that they're orders of magnitude cheaper to create (and destroy) than real, operating system threads. They are precisely around "because concurrency" -- they allow you to nicely model concurrent problems without having to be overly concerned with the implementation detail of their creation cost.

While it's not idiomatic in Go to use a channel/goroutine combo as an iterator, for example -- that's too low-level -- it's absolutely idiomatic to use one for other types of higher-order control flow, managing state machine transitions, doing a scatter/gather, and so on.

Re: Chinese Whispers in Rust

#22

Post by jzelinskie »

pcwalton wrote: post #9

Copying my message on Reddit here: I profiled this and found that the vast majority of the time was spent allocating stack segments. So the basic problem is that this benchmark is simply tuned for segmented stacks. An implementation that does not use segmented stacks will do worse on this benchmark. We've rejected segmented stacks because they don't perform well in the real world (and hurt many other benchmarks, incl…

IIRC I've read something that said Go was going to move away from segmented stacks in future versions. I have no idea about the progress.

Re: Chinese Whispers in Rust

#23

Post by msie »

mistercow wrote: post #19
msie wrote: post #16

I must really be sheltered because that is only the third time I've ever heard the term "Chinese Whispers" used for a game that I remembered as "Broken telephone".

I'm guessing that you're American, and that the author is not. My understanding is that in most of the English-speaking world, the game is called "Chinese whispers" (although that name is falling out of favor because racism), while in America, it is usually called "telephone" or some variation on that.

I'm Canadian so same thing, basically. The term is used by Rob Pike in the referred presentation and he's Canadian, but he's somewhat older than I am. I first noticed the term used by an Australian coworker about a year ago and I also heard it on a Stephen Fry talk show - so yes to your theory.

Re: Chinese Whispers in Rust

#24

Post by [deleted] »

redbad wrote: post #21
coldtea wrote: post #15

Earlier quoted context omitted.

> Oh, come on. Not only is your comment needlessly insulting but it's completely unconstructive. I find his comment pretty spot on. If anything it was the "don't do CSP in Rust" comment that was unconstructive and passive aggresively insulting. Instead of understanding what he read and the constrains it shows, the commenter preffered to just piss on the language. As for the example, it was obviously "retarted" or rat…

> CSP and channels are meant for getting real (cpu > intensive) work done. You don't just use threads (or > greenthreads) "because concurrency". The whole point of green threads is that they're orders of magnitude cheaper to create (and destroy) than real, operating system threads. They are precisely around "because concurrency" -- they allow you to nicely model concurrent problems without having to be overly concern…

[deleted]

Re: Chinese Whispers in Rust

#25

Post by strcat »

redbad wrote: post #21
coldtea wrote: post #15

Earlier quoted context omitted.

> Oh, come on. Not only is your comment needlessly insulting but it's completely unconstructive. I find his comment pretty spot on. If anything it was the "don't do CSP in Rust" comment that was unconstructive and passive aggresively insulting. Instead of understanding what he read and the constrains it shows, the commenter preffered to just piss on the language. As for the example, it was obviously "retarted" or rat…

> CSP and channels are meant for getting real (cpu > intensive) work done. You don't just use threads (or > greenthreads) "because concurrency". The whole point of green threads is that they're orders of magnitude cheaper to create (and destroy) than real, operating system threads. They are precisely around "because concurrency" -- they allow you to nicely model concurrent problems without having to be overly concern…

Rust tasks have the same large fixed-size stack as OS threads. A fine-grained concurrency model like a task graph would be build on top of them.

Re: Chinese Whispers in Rust

#26

Post by pcwalton »

samth wrote: post #14
pcwalton wrote: post #10

Earlier quoted context omitted.

The solution is "don't spawn a huge number of tasks in a tight loop". Rust tasks are optimized for maximum performance once you spin them up, at the cost of some overhead once you spawn them. That was a deliberate choice, because not having segmented stacks makes many much more important things far faster, such as calling into C and performing tight sequential computations without thrashing on stack boundaries or usi…

> In practical usage, we found that the ability to call into C cheaply was much more important than the benefits of small stacks, which mostly help microbenchmarks like this at the expense of real-world code. For Rust's goals, I can definitely believe that the first is true. I don't think the second claim is true at all, though. Lightweight threads have been extremely successful in Haskell, Erlang, etc, and not just…

> I don't think the second claim is true at all, though. Lightweight threads have been extremely successful in Haskell, Erlang, etc, and not just on microbenchmarks.

They're mostly successful in languages that GC/heap-allocate all stack frames (paying the costs that come with it). Rust uses the machine stack so the benefits come with some very significant drawbacks, most notably unpredictability of performance that comes from stack thrashing. Large mallocs are slower than small mallocs in general, but on the typical sizes you need for machine stack segments you probably aren't going to hit the malloc implementation's free list anyway, so you might as well just allocate up front. If you GC stack frames, though, you just bump allocate in the nursery, at the cost of greatly increased GC pressure (but it makes task spawning much cheaper).

There's some interesting discussion on this near the end of Simon Marlow's paper "Extending the Haskell Foreign Function Interface with Concurrency": http://community.haskell.org/~simonmar/papers/conc-ffi.pdf

Re: Chinese Whispers in Rust

#27

Post by strcat »

levosmetalo wrote: post #4

After compiling this with the latest rust built from git, I get only the "Illegal instruction" as an error message when I try to run this example.

You're running out-of-memory, and hitting the `llvm.trap` intrinsic call. It causes an illegal instruction to invoke a core dump. It doesn't currently print an error message.

Re: Chinese Whispers in Rust

#28

Post by kinofcain »

pcwalton wrote: post #10
kinofcain wrote: post #6

Earlier quoted context omitted.

Sadly it seems the "solution" there is "don't do CSP in Rust".

The solution is "don't spawn a huge number of tasks in a tight loop". Rust tasks are optimized for maximum performance once you spin them up, at the cost of some overhead once you spawn them. That was a deliberate choice, because not having segmented stacks makes many much more important things far faster, such as calling into C and performing tight sequential computations without thrashing on stack boundaries or usi…

Thanks for the reply. My comment apparently came off as trite, apologies.

One of the enlightening things for me about CSP and Erlang, in particular, was using channels and messages and a bunch of lightweight tasks for flow control and state management. Reading the reddit comments, and even your reply here, seems to indicate that that type of programming isn't a good fit for rust, and that perfectly reasonable trade offs were made that led the language in a different direction. I should have phrased the comment better, but the recommendations here and in the reddit thread boil down to replying to "it hurts when I do this" with "don't do that".

Obviously no-one is trying to write programs that count to 10001, but the capability of doing it that way efficiently opens up a bunch of different options.

It will be interesting to see what happens to go's performance on things like this if they move away from segmented stacks.

Re: Chinese Whispers in Rust

#29

Post by samth »

pcwalton wrote: post #26
samth wrote: post #14

Earlier quoted context omitted.

> In practical usage, we found that the ability to call into C cheaply was much more important than the benefits of small stacks, which mostly help microbenchmarks like this at the expense of real-world code. For Rust's goals, I can definitely believe that the first is true. I don't think the second claim is true at all, though. Lightweight threads have been extremely successful in Haskell, Erlang, etc, and not just…

> I don't think the second claim is true at all, though. Lightweight threads have been extremely successful in Haskell, Erlang, etc, and not just on microbenchmarks. They're mostly successful in languages that GC/heap-allocate all stack frames (paying the costs that come with it). Rust uses the machine stack so the benefits come with some very significant drawbacks, most notably unpredictability of performance that c…

I think Simon & Simon are making exactly the point I am: lightweight threads are significantly faster than any other threading system, and work in real system, not just microbenchmarks. Rust has made some other design choices that it seems make it harder to have some nice things that most modern functional language runtimes have -- I'm sure there are good reasons for all of those choices. But I don't think your claim that the tradeoffs "mostly help microbenchmarks" is right.

Re: Chinese Whispers in Rust

#30

Post by redbad »

strcat wrote: post #25
redbad wrote: post #21

Earlier quoted context omitted.

> CSP and channels are meant for getting real (cpu > intensive) work done. You don't just use threads (or > greenthreads) "because concurrency". The whole point of green threads is that they're orders of magnitude cheaper to create (and destroy) than real, operating system threads. They are precisely around "because concurrency" -- they allow you to nicely model concurrent problems without having to be overly concern…

Rust tasks have the same large fixed-size stack as OS threads. A fine-grained concurrency model like a task graph would be build on top of them.

    > Rust tasks have the same large fixed-size stack as OS 
    > threads. A fine-grained concurrency model like a task 
    > graph would be build on top of them.
In the absence of other context (I don't really know much about Rust) I would then argue that Rust tasks miss the point of CSP.
Post reply on HN