Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is parallel programming hard, and, if so, what can you do about it?

mirrors.edge.kernel.org

Post reply on HN
Print view View original

41–50 of 199 posts

Re: Is parallel programming hard, and, if so, what can you do about it?

#41

Post by stevage »

cj wrote: post #23

This reminds me when I was going through the YC accelerator in 2012. We were building a web-based email client, and PG didn’t like the idea. He pulled our team aside during one of the batch-wide Tues night dinners and suggested we pivot to building something that could take single threaded programs and quickly/easily make them multi-threaded. No one on our team knew anything about threading (none of us had even gradu…

That's such a random suggestion. To switch from building a user application to building a tool for devs...that is in no way related to what you were working on?

Re: Is parallel programming hard, and, if so, what can you do about it?

#42

Post by jabl »

cj wrote: post #23

This reminds me when I was going through the YC accelerator in 2012. We were building a web-based email client, and PG didn’t like the idea. He pulled our team aside during one of the batch-wide Tues night dinners and suggested we pivot to building something that could take single threaded programs and quickly/easily make them multi-threaded. No one on our team knew anything about threading (none of us had even gradu…

OTOH PG suggesting that some young people with zero experience with multithreading pivot to searching for the grail is, just, daft. A topic which computer science PhD's have failed to find a solution to, despite decades of research. Not sure the world needed another web-based email client, but at least that project had some non-zero chance of success.

Re: Is parallel programming hard, and, if so, what can you do about it?

#43

Post by kllrnohj »

vmfunction wrote: post #32
Const-me wrote: post #24

As a developer, I often choose higher-level APIs not listed in that article. On Windows, OSX and iOS the OS userland already implements general, and relatively easy to use, thread pools. On Windows, see CreateThreadpoolWork, WaitForThreadpoolWorkCallbacks, etc. It’s easier to use threads with locks while someone else is managing these threads. On Apple, the pool is called “grand central dispatch” and does pretty much…

what about "green threads" that is not managed by the OS like https://tokio.rs ?

You only ever need this if you're trying to have hundreds of thousands to millions of threads. It's a very niche problem to have

Re: Is parallel programming hard, and, if so, what can you do about it?

#44

Post by DizzyDoo »

jetbalsa wrote: post #37
remontoire wrote: post #27

Watching geohot code a general matrix multiply algorithm from 0.9 GFLOPS and optimising it to 100 glops by only tinkering with cache locality, it makes me wonder how much effort should be put into single threaded performance before ever thinking about multi threading

I've seen stuff like that before with a game called Factroio, The only game I've ever see that is optimized so hard that your RAM Speed can affect large bases rather quickly, same with faster L2 Cache. Their entire blog series[1] covers a large part of how they did this. but for a game written mostly in LUA they sure did a good job on it. 1: https://www.factorio.com/blog/post/fff-204

Yes, the Factorio devs had an approach where they optimised everything happening in the game in the original singlethreaded environment, before moving onto multithreaded support. That's where the game is now, and as far as I understand it the multithreading occurs on each independent set of conveyor belts or belt lanes, and there's some info on that in this blog post[0] for anyone interested.

[0] - https://www.factorio.com/blog/post/fff-364

Re: Is parallel programming hard, and, if so, what can you do about it?

#45

Post by jabl »

aleph_minus_one wrote: post #39
vmfunction wrote: post #32

Earlier quoted context omitted.

what about "green threads" that is not managed by the OS like https://tokio.rs ?

Green threads do not make use of multiple cores of a modern processor.

It's entirely possible for a green threads implementation to schedule the green threads over multiple OS threads, thus making use of multiple cores. The programming language "Go" being a popular implementation of this.

Re: Is parallel programming hard, and, if so, what can you do about it?

#46

Post by Const-me »

vmfunction wrote: post #32
Const-me wrote: post #24

As a developer, I often choose higher-level APIs not listed in that article. On Windows, OSX and iOS the OS userland already implements general, and relatively easy to use, thread pools. On Windows, see CreateThreadpoolWork, WaitForThreadpoolWorkCallbacks, etc. It’s easier to use threads with locks while someone else is managing these threads. On Apple, the pool is called “grand central dispatch” and does pretty much…

what about "green threads" that is not managed by the OS like https://tokio.rs ?

I believe these green threads work well when there’s good support in both language, runtime and standard library. I have built complicated concurrent software in C# with async-await. I don’t program golang but I heard the concurrency model works rather well in golang too, probably for the same reason as C#: good support in the language and the runtime.

I have no idea about Tokyo. I don’t program Rust, and the feedback I read about async/await was mixed.

Re: Is parallel programming hard, and, if so, what can you do about it?

#47

Post by z3t4 »

The symtoms that lead up to me learning about the need for locks in multi threaded servers was painful. That the state could change from one line of code to the next line, how could it change when I just set it a micro second ago? Because the user clicked the button twice and the call was executed simultaneously but in different threads because my new server had many cores. Then I learned about async programming, but I no longer remember what was so difficult about it. Just that it took about a year and I basically had to rewire how my brain visualize program execution.

Re: Is parallel programming hard, and, if so, what can you do about it?

#48

Post by tomrod »

tambourine_man wrote: post #38
klabb3 wrote: post #5

I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?

Stranded on a deserted island is bit radical, but I recommend going to a place with no internet connection for a week or so. I though I had a long reading problem. Turns out, I have an internet problem.

I didn't realize I also was a tambourine_man -- you literally just described my recent realization.

I'm moving this week and won't have internet for a few days. I look forward to the relative break.

Re: Is parallel programming hard, and, if so, what can you do about it?

#49

Post by karelpeeters »

vmfunction wrote: post #32
Const-me wrote: post #24

As a developer, I often choose higher-level APIs not listed in that article. On Windows, OSX and iOS the OS userland already implements general, and relatively easy to use, thread pools. On Windows, see CreateThreadpoolWork, WaitForThreadpoolWorkCallbacks, etc. It’s easier to use threads with locks while someone else is managing these threads. On Apple, the pool is called “grand central dispatch” and does pretty much…

what about "green threads" that is not managed by the OS like https://tokio.rs ?

Tokio is using the Rust async features, which are not green threads. In the former code has to explicitly mark potential yield points, in the latter green threads can be scheduled by the runtime without any help from the code itself.

As a historical note, Rust used to have green threads but they were abandoned a long time ago. This is a good talk about both the differences between different forms of concurrency/async and Rusts history with them: https://www.infoq.com/presentations/rust-2019/ (includes a transcript)

Re: Is parallel programming hard, and, if so, what can you do about it?

#50

Post by cpach »

PheonixPharts wrote: post #30
davidgrenier wrote: post #20

This looks brutal, for a lot of people John Reppy's book Concurrent Programming in ML (as in SML not Machine Learning) is going to be much more accessible. Pick the CSP-style library in the programming language of your choice. Go with goroutines and channels Clojure with core.async F# with Hopac It would be a very interesting project to roll your own in C# using Microsoft Robotics Studio's CCR (Coordination and Concu…

There are multiple replies like this one, but it's a bit shocking to see that on Hacker News people don't know the difference between concurrent programming and parallel programming. Concurrency means that you can have multiple tasks running in the same time period, Parallelism means you have multiple tasks running at the same time . The most obvious demonstration of this is that you can (and many languages do) have…

Correct me if I’m wrong, but isn’t concurrency enough for “most” use-cases? When does one really need true parallelism?
Post reply on HN