Is parallel programming hard, and, if so, what can you do about it?

Post by crabbone » Wed, Jun 14, 2023, 3:59 PM UTC

ascar wrote: post #101

vivegi wrote: post #94

Earlier quoted context omitted.

The book covers it in Appendix A.6 (p 424) in the v2023.06.11a PDF file. > A.6 What is the Difference Between “Concurrent” and “Parallel”? > From a classic computing perspective, “concurrent” and “parallel” are clearly synonyms. However, this has not stopped many people from drawing distinctions between the two, and it turns out that these distinctions can be understood from a couple of different perspectives. > The…

Well, funnily enough this does read in contrast to the definitions used in Wikipedia, which are the ones I am also familiar with (I also do teach a class called "Parallel Programming" to graduates). I do think the differentation make sense from a perspective of problem classes, as also evident from the comments here. Running independent problems in parallel to better utilize hardware ressources is very different from…

Post by gpderetta » Wed, Jun 14, 2023, 4:01 PM UTC

mrkeen wrote: post #92

z3t4 wrote: post #47

The symtoms that lead up to me learning about the need for locks in multi threaded servers was painful. That the state could change from one line of code to the next line, how could it change when I just set it a micro second ago? Because the user clicked the button twice and the call was executed simultaneously but in different threads because my new server had many cores. Then I learned about async programming, but…

> the state could change from one line of code to the next line > I no longer remember what was so difficult about it You or (your libraries) probably stopped mutating things. No mutation means you can rely on values you used several lines ago. This is the perennial HN discussion whenever the word 'functional' comes up. "But x=x+1 is simple to teach to beginners" is the rallying cry. Well, this is why not. Also, > th…

Post by dahart » Wed, Jun 14, 2023, 4:03 PM UTC

ascar wrote: post #84

I see a lot of confusion between parallel programming [1] and concurrent programming [2] in the comments here. The former and what this book is about deals with the problem of parallelizaing a single sequential program. There usually is strong interaction or dependencies between elements and progress needs synchronization. E.g. timestep iterations in real-time simulations that need synchronization with data communica…

Post by Nathanba » Wed, Jun 14, 2023, 4:09 PM UTC

phkahler wrote: post #114

crabbone wrote: post #106

Earlier quoted context omitted.

I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was released. So, I associate this nonsense with Go, but it could have happened earlier, I simply never heard about it before then. Anyways. The way I see it used today, it's applied to language runtimes incapable or severely crippled when it comes to parallel / conc…

>> don't know who invented this nonsense distinction ... It not nonsense. In C or C++ lots of code can be made parallel using OpenMP and inserting some #pragma statements above for loops. This does not work for things like running a UI in one thread and some other work in another thread, perhaps displaying results as they are found. These are quite different types of parallelism.

Post by qznc » Wed, Jun 14, 2023, 4:11 PM UTC

mrkeen wrote: post #104

twoodfin wrote: post #53

Earlier quoted context omitted.

You might be interested in Joe Duffy’s retrospective on Microsoft’s failed experiment to integrate STM into .NET: https://joeduffyblog.com/2010/01/03/a-brief-retrospective-on...

You really only need one paragraph from that: > What do we do with atomic blocks that do not simply consist of pure memory reads and writes? (In other words, the majority of blocks of code written today.) If you could just get programmers to stop mutating, you could get STM (which incidentally would give you back the ability to mutate)