Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is parallel programming hard, and, if so, what can you do about it?

mirrors.edge.kernel.org

Post reply on HN
Print view View original

11–20 of 199 posts

Re: Is parallel programming hard, and, if so, what can you do about it?

#11

Post by tm-guimaraes »

RyanHamilton wrote: post #8

>>Is Parallel Programming Hard, and, If So, What Can You Do About It? I confess I didn't read TFA but the title made me think of something that hopefully some people here are well placed to take further. In a programming language I use there is an "each" function. For example in pseudo-java given a function f(int i). result = f each List . Runs f over each integer. To make this multi-threaded you can simply say f pea…

So instead we would have ridiculous amount of problems with cache and scheduling. I bet most of the time sequential cose would be “faster”, due to the parallel overheads, the cache misses, and the scheduling that obviously can’t fit every loop scenario.

And this is assuming that the code in the loop is thread safe, otherwise you will summon every concurrency bug on the planet.

Re: Is parallel programming hard, and, if so, what can you do about it?

#13

Post by tm-guimaraes »

flohofwoe wrote: post #10
RyanHamilton wrote: post #8

>>Is Parallel Programming Hard, and, If So, What Can You Do About It? I confess I didn't read TFA but the title made me think of something that hopefully some people here are well placed to take further. In a programming language I use there is an "each" function. For example in pseudo-java given a function f(int i). result = f each List . Runs f over each integer. To make this multi-threaded you can simply say f pea…

It's only easy if the items that are iterated over are entirely self contained, and the code in the loop body also does not need (write-)access to any other state that might be shared between threads. Rust has such limitations built into the language, but this is also one of the points that make Rust "difficult" (because you actually need to think first about how you organize your data to be "multi-threading-compatib…

That’s only safety. It still does not guarantee that performance would improve.

Re: Is parallel programming hard, and, if so, what can you do about it?

#14

Post by ascar »

samsquire wrote: post #2

Thanks for another thing I need to add to my reading list in addition to The Art of Multiprocessor Programming. I am really interested in parallel, asynchronous, multithreading, coroutine, futures programming so it's what I spend my days thinking about and blogging about it. I hope you sense my excitement in this comment about this topic. I'm looking for a programming model that parallelises easily and doesn't requir…

> I am really interested in parallel, asynchronous, multithreading, coroutine, futures programming so it's what I spend my days thinking about and blogging about it.

That's not parallel programming though. Parallell programming deals with the parallelization of a single sequential algorithm or program (e.g. weather simulation) across multiple threads, CPU or machines, usually with the requirement of real time synchronization. When you paralellize independent tasks (async, coroutines, futures) a whole lot of problems just don't exist and others go more into focus.

Re: Is parallel programming hard, and, if so, what can you do about it?

#15

Post by ascar »

klabb3 wrote: post #5

I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?

> concurrent programming

If you're interested in concurrent programming this book won't give you much. The topics focus on the parallelization of a sequential algorithm and go into detail about things like synchronization (locking, barriers), important HW details (like caches and CPU pipelining) and algorithmic approaches. Imagine a weather simulation and not concurrent requests to a web server.

Re: Is parallel programming hard, and, if so, what can you do about it?

#16

Post by samsquire »

ascar wrote: post #14
samsquire wrote: post #2

Thanks for another thing I need to add to my reading list in addition to The Art of Multiprocessor Programming. I am really interested in parallel, asynchronous, multithreading, coroutine, futures programming so it's what I spend my days thinking about and blogging about it. I hope you sense my excitement in this comment about this topic. I'm looking for a programming model that parallelises easily and doesn't requir…

> I am really interested in parallel, asynchronous, multithreading, coroutine, futures programming so it's what I spend my days thinking about and blogging about it. That's not parallel programming though. Parallell programming deals with the parallelization of a single sequential algorithm or program (e.g. weather simulation) across multiple threads, CPU or machines, usually with the requirement of real time synchro…

What would you call what I'm working on, it is "parallel" but probably doesn't come under "parallel programming" in the literature. I'm still interested in it though, just haven't got around to it yet, it's a large space.

I'm not working on novel parallel algorithms that solve computer science problems, I am interested in parallelism as a general principle for coordinating systems activities.

When I get time to delve into parallel algorithms then I'll work on that, at the moment there's a lot of work in just cordinating and scheduling parallel tasks.

I am also interested in this area of database internals such as parallel query execution and concurrency control and wait/lockfree algorithms

I feel it is complicated and wide space and we deceive ourselves to think we know everything or hubris, which I expressly avoid doing.

Re: Is parallel programming hard, and, if so, what can you do about it?

#17

Post by jayjader »

RyanHamilton wrote: post #8

>>Is Parallel Programming Hard, and, If So, What Can You Do About It? I confess I didn't read TFA but the title made me think of something that hopefully some people here are well placed to take further. In a programming language I use there is an "each" function. For example in pseudo-java given a function f(int i). result = f each List . Runs f over each integer. To make this multi-threaded you can simply say f pea…

I don't know of any programming language that does that as a language-first primitive. But this is basically slapping a `#pragma openmp parallel for` before your C for loop. In Rust there is the crate `rayon` that takes advantage of the language-level trait system to "augment" any (existing) iterable with a `par_iter()` method that effectively does what you describe: when included in the same module you can simply `for foo in bar.par_iter()` in place of `for foo in bar.iter()` to parallelize your iterations.

I suspect any language wanting to offer this as the default behavior for `for` loops will end up being a language like Java, Erlang, or SmallTalk - the language spec includes a "virtual" runtime that allows it to assume such flexibility in behavior, and capacity to "create/run threads/processes" that otherwise requires an OS (as in, you start leaving the perimeter of a programming language).

Re: Is parallel programming hard, and, if so, what can you do about it?

#18

Post by KeplerBoy »

klabb3 wrote: post #5

I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?

There's no need to read the whole thing. Read a chapter you're interested in and go back to the book once you're interested in another chapter. It's not a novel.

Re: Is parallel programming hard, and, if so, what can you do about it?

#19

Post by jefc1111 »

My answers to the two questions in the post title: "Yes" and "Read Concurrent Data Processing in Elixir [1]"

I have dealt with concurrency in PHP (good times with Laravel Horizon) in a couple of different ways and in NodeJS (never feels good, to me). The BEAM has some great primitives for concurrency and the book mentioned above walks the reader through them at a good pace. I read it cover to cover and it really enriched my mental model of how to use concurrency and async.

[1] https://pragprog.com/titles/sgdpelixir/concurrent-data-proce...

Re: Is parallel programming hard, and, if so, what can you do about it?

#20

Post by davidgrenier »

This looks brutal, for a lot of people John Reppy's book Concurrent Programming in ML (as in SML not Machine Learning) is going to be much more accessible. Pick the CSP-style library in the programming language of your choice.

Go with goroutines and channels

Clojure with core.async

F# with Hopac

It would be a very interesting project to roll your own in C# using Microsoft Robotics Studio's CCR (Coordination and Concurrency Runtime) (though I speculate those are buffered channels by default).

Post reply on HN