Live data from Hacker News

The free lunch is over: a fundamental turn toward concurrency in software (2005)

gotw.ca

31–40 of 101 posts

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#31
post #30
post #26

Earlier quoted context omitted.

I find the distinction far less interesting than most people. I thing it's easier to think of parallelism simply as an interesting special case of concurrency, and to think of runtimes and systems that are "concurrent" but can't run literally simultaneously, like Javascript or Python, as simply accidents of history not worth specially writing into the definitions of our terms. Every year "concurrent" code that can't…

What's your definition of running simultaneously because CPython's approach to running on multiple cores is multiprocessing which works honestly fine. The tooling to do it is pretty slick where you can ignore a lot of the typical pain of IPC. Because if "good" concurrency support means single-process multi-threaded on multiple cores running with enough locks that you can have multiple threads executing code simultane…

Shared memory spaces. Your "slick tooling" is one of the hacks I mentioned that history will forget, by the standard of "if a new language emerged that tried to call that 'concurrency' nobody would take it seriously".

I should say that "hack" here isn't necessarily a perjorative. There are reasons for communities to create and deploy those. There are plenty of cases where existing code can be leveraged to work better than it could without it, and that's the relevant standard for whether something is useful, not whether or not in a parallel universe the code could have been written in some completely different way or whether in some hypothetical sense if you could push a button and rewrite something for free you'd end up with something better. Hacks can be good, and every language will pick them up at some point as history evolves around it and some of the core assumptions a language/runtime made go out of date. But it's still a hack.

While OS process boundaries do provide some nice features, they are also in general overkill. See Erlang and Pony for some alternate riffs on the idea, and if you look at it hard enough and understand it deeply enough, even what Rust does can provide some similar benefits without raising a full OS process boundary between bits of code.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#32
post #23

There seems to be no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores. Nondeterminism makes parallelism and concurrency inherently hard and unfriendly to new comers. It becomes even more difficult in recent years due to architecture decisions: weak memory order makes things worse. Supposing you are going to write nontrivial concurrent programs like toy Raft, I believe that lo…

> There seems to be no way to efficiently replay concurrent programs in a deterministic fashion

I wish there was something like Jepsen [1] broadly available to mainstream programming languages to do just that.

[1] https://jepsen.io/consistency

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#33
post #23

There seems to be no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores. Nondeterminism makes parallelism and concurrency inherently hard and unfriendly to new comers. It becomes even more difficult in recent years due to architecture decisions: weak memory order makes things worse. Supposing you are going to write nontrivial concurrent programs like toy Raft, I believe that lo…

> no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores

OpenMP with static scheduler and hardcoded count of threads?

OpenMP ain’t a silver bullet, but for many practically useful scenarios it’s indeed as simple as calling a function.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#34
post #9

If only GPUs were more common and easier to work with.

> If only GPUs were more common

They are very common.

In 2021 on Windows, is safe to assume the GPU does at least D3D 11.0. The last GPU which does not is Intel Sandy Bridge, discontinued in 2013. D3D 11.0 supports lots of features about GPGPU. The main issue with D3D11 is FP64 support being optional, the hardware may or may not support.

> and easier to work with

I don’t find neither DirectCompute 11, nor CUDA, exceptionally hard to work with. D3D12 and Vulkan are indeed hard, IMO.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#35
post #26
post #16

Earlier quoted context omitted.

You mean parallelism ? JS has good concurrency support (event, async..)

I find the distinction far less interesting than most people. I thing it's easier to think of parallelism simply as an interesting special case of concurrency, and to think of runtimes and systems that are "concurrent" but can't run literally simultaneously, like Javascript or Python, as simply accidents of history not worth specially writing into the definitions of our terms. Every year "concurrent" code that can't…

I have usually seen the distinction between concurrency and parallelism being drawn for precisely the opposite reason: parallelism without concurrency is a relatively commonly used niche, it is much simpler than concurrency, and it has special tools that make sense just for it.

For example, CUDA exposes a parallel programming model with low support for actual concurrency. Similarly, OpenMP is mostly useful for parallel computations that rarely need coordination. C#'s Parallel package is another example.

By contrast, concurrency often rears its ugly head even in single-threaded workflows, even ones that are part of larger multi-threaded programs.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#36
post #31
post #30

Earlier quoted context omitted.

What's your definition of running simultaneously because CPython's approach to running on multiple cores is multiprocessing which works honestly fine. The tooling to do it is pretty slick where you can ignore a lot of the typical pain of IPC. Because if "good" concurrency support means single-process multi-threaded on multiple cores running with enough locks that you can have multiple threads executing code simultane…

Shared memory spaces. Your "slick tooling" is one of the hacks I mentioned that history will forget, by the standard of "if a new language emerged that tried to call that 'concurrency' nobody would take it seriously". I should say that "hack" here isn't necessarily a perjorative. There are reasons for communities to create and deploy those. There are plenty of cases where existing code can be leveraged to work better…

You're saying that shared memory concurrency is the future? I think you've got it completely wrong. Shared memory concurrency is the past. It was thought to be a good idea in the 80s and 90s, but we now know that it's both hard to program, and that it works poorly with highly-parallel, high-performance hardware. In the future, we'll see more and more programming systems which don't provide shared memory concurrency at all.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#37
I find it interesting that the language that best deals with parallelism (IMO) was invented significantly before we moved in the multi-core direction.

The programming language landscape evolved and developed in the face of multi-code - async being a classic example. But the language that's often held up as the best solution to any given parallelism problem is Erlang. Erlang was built as a good programming model for concurrency on a single core, and then when multi-core came along, SMP was 'just' a VM enhancement and no programs needed changing at all to get the full advantage of all the cores (barring some special cases).

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#38
post #9

If only GPUs were more common and easier to work with.

> If only GPUs were more common They are very common. In 2021 on Windows, is safe to assume the GPU does at least D3D 11.0. The last GPU which does not is Intel Sandy Bridge, discontinued in 2013. D3D 11.0 supports lots of features about GPGPU. The main issue with D3D11 is FP64 support being optional, the hardware may or may not support. > and easier to work with I don’t find neither DirectCompute 11, nor CUDA, excep…

> I don’t find neither DirectCompute 11, nor CUDA, exceptionally hard to work with.

It still sucks that, if you're not careful or unlucky, GPUs may interfere with normal video operation during setup.

Also on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine.

Further, the form factor sucks. Every card comes with 4 or more DVI connectors which I don't need.

If parallel is the future, then give me something that fits on the mainboard directly, and which is more or less a commodity. Not the proprietary crap that vendors are currently shoving at us.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#39
post #23

There seems to be no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores. Nondeterminism makes parallelism and concurrency inherently hard and unfriendly to new comers. It becomes even more difficult in recent years due to architecture decisions: weak memory order makes things worse. Supposing you are going to write nontrivial concurrent programs like toy Raft, I believe that lo…

> no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores OpenMP with static scheduler and hardcoded count of threads? OpenMP ain’t a silver bullet, but for many practically useful scenarios it’s indeed as simple as calling a function.

what about if you throw in async IO into the mix? seems like you'd have to have some kind of vmware-like virtualization where you record all IO interrupts into a linearalizable log

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#40
post #16
post #15

I blame Python and Javascript. Two of the most popular languages without proper concurrency support.

You mean parallelism ? JS has good concurrency support (event, async..)

Parallelism in JavaScript is simple using web workers. of course the tricky part as with all parallel applications is managing the complexity you create yourself. The only languages that seem to do a good job at handling this for you are challenging in other ways like Haskell and Erlang.
Post reply on HN