Live data from Hacker News

Is parallel programming hard, and, if so, what can you do about it?

mirrors.edge.kernel.org

21–30 of 199 posts

Re: Is parallel programming hard, and, if so, what can you do about it?

#21
post #15
post #5

I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?

> concurrent programming If you're interested in concurrent programming this book won't give you much. The topics focus on the parallelization of a sequential algorithm and go into detail about things like synchronization (locking, barriers), important HW details (like caches and CPU pipelining) and algorithmic approaches. Imagine a weather simulation and not concurrent requests to a web server.

Good to know! Yes I’m less interested in “the art of programming but parallel” (partly because I don’t have PhD level ambitions) and more interested in how to effectively program day-to-day boring stuff that also needs concurrency.

My assessment is that the latter is nowhere near “solved” (hesitant to use that word because craft unlike eg proofs is about trade offs), in the sense that concurrency differs a lot across our tools (languages, runtimes etc) and it even differs quite substantially within the main paradigms, like coroutines, async, green threads, native threading etc.

If we compare with other advanced language features like say memory management, we’ve come much further, imo (GC, RAII, ref counting, manual are pretty much all well understood as well as the stack-heap duality, adopted basically universally). With concurrency we have, if we’re being generous, merely mutices as the common ground. Even “simple” notification mechanisms and ownership transfer across tasks would vary greatly across languages and often be quite contrived.

Re: Is parallel programming hard, and, if so, what can you do about it?

#23
This reminds me when I was going through the YC accelerator in 2012.

We were building a web-based email client, and PG didn’t like the idea. He pulled our team aside during one of the batch-wide Tues night dinners and suggested we pivot to building something that could take single threaded programs and quickly/easily make them multi-threaded.

No one on our team knew anything about threading (none of us had even graduated college). So we kept working on the email client, now defunct.

PG brought us in a back room where the Stripe founders were waiting (I think they were speaking that night). We were brainstorming ideas with them, and one of the brothers recommended we build something having to do with phones and calling people (don’t remember the idea anymore).

PG was not very happy when he heard we weren’t going to pivot and kept on working on a new email client :)

Re: Is parallel programming hard, and, if so, what can you do about it?

#24
As a developer, I often choose higher-level APIs not listed in that article.

On Windows, OSX and iOS the OS userland already implements general, and relatively easy to use, thread pools. On Windows, see CreateThreadpoolWork, WaitForThreadpoolWorkCallbacks, etc. It’s easier to use threads with locks while someone else is managing these threads. On Apple, the pool is called “grand central dispatch” and does pretty much the same thing.

Modern Windows kernel supports interesting synchronization APIs like WaitOnAddress, WakeByAddressSingle which allow to implement locks without the complexity or performance overhead of maintaining special synchronization objects.

Linux kernel implements performant and scalable message queues, see mq_overview(7). And it has synchronization objects like eventfd() and pidfd_open() which allow to integrate locks or other things into poll/epoll based event loops.

Re: Is parallel programming hard, and, if so, what can you do about it?

#25

This looks brutal, for a lot of people John Reppy's book Concurrent Programming in ML (as in SML not Machine Learning) is going to be much more accessible. Pick the CSP-style library in the programming language of your choice. Go with goroutines and channels Clojure with core.async F# with Hopac It would be a very interesting project to roll your own in C# using Microsoft Robotics Studio's CCR (Coordination and Concu…

Etmology of ML: "Meta Language"

Re: Is parallel programming hard, and, if so, what can you do about it?

#26
post #5

I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?

Check out the table of contents. I did a quick flip through and realized that I'm never going to be doing low level multithreading, so I don't need to deal with OS layer stuff. There were some other ideas too, might be worth flipping to relevant areas. Heck, if I do end up using it, I'll likely read a summary when I'm dealing with it. There are some ideas that I havent heard of, which was alright, but again, since my…

I do a lot of it, but never on the low-end.

99% of mine, is responding to closures for network events and device responses.

When you do that, synchronizing is one way to deal with things (wait for the other thing to finish), or completion testing (is the thing ready for the next step?). Basically, they are the same thing.

You are also not always guaranteed a standard context, but modern languages make it easy to hook to one. In the "old days," we used to use RefCons (Reference Context hooks).

Re: Is parallel programming hard, and, if so, what can you do about it?

#27
Watching geohot code a general matrix multiply algorithm from 0.9 GFLOPS and optimising it to 100 glops by only tinkering with cache locality, it makes me wonder how much effort should be put into single threaded performance before ever thinking about multi threading

Re: Is parallel programming hard, and, if so, what can you do about it?

#28
post #23

This reminds me when I was going through the YC accelerator in 2012. We were building a web-based email client, and PG didn’t like the idea. He pulled our team aside during one of the batch-wide Tues night dinners and suggested we pivot to building something that could take single threaded programs and quickly/easily make them multi-threaded. No one on our team knew anything about threading (none of us had even gradu…

If you had something in 2012 that could magically convert a single threaded program to a parallel one, it would be pretty useful. Seems incredibly difficult to build, though.

Re: Is parallel programming hard, and, if so, what can you do about it?

#29
It is not hard if you use proper libraries. So Take a look into Scala ZIO library, you can solve any concurrency problem with an ease. With the proper toolset which ZIO provides, you can tackle any consurrency problem in typesafe way, and mostly correct if it compiles :)

Re: Is parallel programming hard, and, if so, what can you do about it?

#30

This looks brutal, for a lot of people John Reppy's book Concurrent Programming in ML (as in SML not Machine Learning) is going to be much more accessible. Pick the CSP-style library in the programming language of your choice. Go with goroutines and channels Clojure with core.async F# with Hopac It would be a very interesting project to roll your own in C# using Microsoft Robotics Studio's CCR (Coordination and Concu…

There are multiple replies like this one, but it's a bit shocking to see that on Hacker News people don't know the difference between concurrent programming and parallel programming.

Concurrency means that you can have multiple tasks running in the same time period, Parallelism means you have multiple tasks running at the same time.

The most obvious demonstration of this is that you can (and many languages do) have single threaded concurrency.

I did some grad course work in parallel programming and there's really no way to not make it "brutal", because to really do it in a way that increases performance you need to really understand some low-level performance issues.

Post reply on HN