Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The free lunch is over: a fundamental turn toward concurrency in software (2005)

gotw.ca

Post reply on HN
Print view View original

41–50 of 101 posts

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#41

Post by bottled_poe »

catern wrote: post #36
jerf wrote: post #31

Earlier quoted context omitted.

Shared memory spaces. Your "slick tooling" is one of the hacks I mentioned that history will forget, by the standard of "if a new language emerged that tried to call that 'concurrency' nobody would take it seriously". I should say that "hack" here isn't necessarily a perjorative. There are reasons for communities to create and deploy those. There are plenty of cases where existing code can be leveraged to work better…

You're saying that shared memory concurrency is the future? I think you've got it completely wrong. Shared memory concurrency is the past. It was thought to be a good idea in the 80s and 90s, but we now know that it's both hard to program, and that it works poorly with highly-parallel, high-performance hardware. In the future, we'll see more and more programming systems which don't provide shared memory concurrency a…

I believe there will always be situations (though specific) where a shared-memory concurrency model is the most efficient and performant use of available resources. For this reason alone, shared-memory concurrency will always have a place. That said, I generally agree that the isolated parallel memory model is preferable for simplicity's sake.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#42

Post by Const-me »

amelius wrote: post #38
Const-me wrote: post #34

Earlier quoted context omitted.

> If only GPUs were more common They are very common. In 2021 on Windows, is safe to assume the GPU does at least D3D 11.0. The last GPU which does not is Intel Sandy Bridge, discontinued in 2013. D3D 11.0 supports lots of features about GPGPU. The main issue with D3D11 is FP64 support being optional, the hardware may or may not support. > and easier to work with I don’t find neither DirectCompute 11, nor CUDA, excep…

> I don’t find neither DirectCompute 11, nor CUDA, exceptionally hard to work with. It still sucks that, if you're not careful or unlucky, GPUs may interfere with normal video operation during setup. Also on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine. Further, the form factor sucks. Every card comes with 4 or…

> on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine.

If that’s your computer, just change the registry setting disabling the TDR. By default, Windows uses 2 seconds to limit max.pipeline latency. When exceeded, the OS resets the GPU, restarts the driver, and logs that “device lost” in the event log.

If that’s your customer’s computer you have to be more creative and fix your compute shaders. When you have a lot of things to compute and/or you detect underpowered hardware, split your Dispatch() calls into a series of smaller ones. As a nice side effect, this minimizes effect of your application on 3D rendering things running on the same GPU by other programs.

Don’t submit them all at once. Submit two initially then one at a time using ID3D11Query to track completion of these compute shaders. ID3D11Fence can do that better (can sleep on WaitForSingleObject saving electricity) but fences are less compatible unfortunately. That’s an optional feature introduced in Win10 post-release in some update, and e.g. VMWare SVGA 3D doesn’t support fences.

> the form factor sucks. Every card comes with 4 or more DVI connectors which I don't need.

Some datacenter GPUs, and newer mining GPUs come without connectors. The mainstream commodity cards to have connectors because the primary market for them is supposed to be gamers.

> something that fits on the mainboard directly

Unlikely to happen because thermals. Modern high-end GPUs consume more electricity than comparable CPUs, e.g. RTX 3090 needs 350W, similarly priced EPYC 7453 needs 225W.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#43

Post by [deleted] »

rkangel wrote: post #37

I find it interesting that the language that best deals with parallelism (IMO) was invented significantly before we moved in the multi-core direction. The programming language landscape evolved and developed in the face of multi-code - async being a classic example. But the language that's often held up as the best solution to any given parallelism problem is Erlang. Erlang was built as a good programming model for c…

[deleted]

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#44

Post by Const-me »

hderms wrote: post #39
Const-me wrote: post #33

Earlier quoted context omitted.

> no way to efficiently replay concurrent programs in a deterministic fashion on multiple cores OpenMP with static scheduler and hardcoded count of threads? OpenMP ain’t a silver bullet, but for many practically useful scenarios it’s indeed as simple as calling a function.

what about if you throw in async IO into the mix? seems like you'd have to have some kind of vmware-like virtualization where you record all IO interrupts into a linearalizable log

When you throw async I/O in the mix it’s no longer just your application you’re debugging, it’s the whole OS with other processes, the OS kernel with these drivers and DMAs, and external hardware that’s not a code at all.

Can be tricky to debug or implement, but when I need that I normally use async-await in C#. The debugger in the IDE is multi-threaded, and does support these tasks.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#45

Post by tuyiown »

spacechild1 wrote: post #6
rustybolt wrote: post #2

People having been saying this for decades and while it's true, concurrency is still widely regarded as 'too hard'. I'm not sure if this is justified (e.g. concurrency is inherently too hard to be viable), or due to the lack of tooling/conventions/education.

> concurrency is still widely regarded as 'too hard'. The question is: by whom? High performance software, like game engines, DAWs or video editors, has been heavily multithreaded for a while now. Maybe it's consumer or business software that could profit from more multithreading? I don't know, because I don't work in those areas.

The hard part is to improve perceived performance or resource overhead for general software, where each task outcome is tightly coupled with app state, eg. a parallelize a slow task on button click and have guarantee that final state is coherent. Nothing out of reach, but doing that without compromising too much on maintainability is still an opened question.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#46

Post by amelius »

Const-me wrote: post #42
amelius wrote: post #38

Earlier quoted context omitted.

> I don’t find neither DirectCompute 11, nor CUDA, exceptionally hard to work with. It still sucks that, if you're not careful or unlucky, GPUs may interfere with normal video operation during setup. Also on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine. Further, the form factor sucks. Every card comes with 4 or…

> on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine. If that’s your computer, just change the registry setting disabling the TDR. By default, Windows uses 2 seconds to limit max.pipeline latency. When exceeded, the OS resets the GPU, restarts the driver, and logs that “device lost” in the event log. If that’s your…

Thanks for the advice (I'm on Linux), but my point is that the GPU really is not the "first-class citizen" that the CPU is.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#47

Post by ashleyn »

rkangel wrote: post #37

I find it interesting that the language that best deals with parallelism (IMO) was invented significantly before we moved in the multi-core direction. The programming language landscape evolved and developed in the face of multi-code - async being a classic example. But the language that's often held up as the best solution to any given parallelism problem is Erlang. Erlang was built as a good programming model for c…

The writing was on the wall as early as the beginning of the Pentium 4 era.

There was this assumption that clock speed would just continue to grow at the insane rates it did in the 80s and 90s, and then reality caught up when the Netburst architecture didn't scale well enough to avoid heat issues and crazy insane pipelines. When Intel rolled out the Core architecture, they went back to a P6-derived design, a microarchitecture that dated back to 1995 - effectively an admission that Netburst had failed.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#48

Post by flakiness »

2005, when the most important platform was the desktop, when the virtualization was yet to be mature, and when the dominant system programming language is C++.

Today CPU cores are sliced by cloud vendors to sell out in smaller portion, and the phones are hesitant to go many-core as it will eat your battery in light-speed. Dark silicon is spent for domain specific circuits like AI, media or networking instead of generic cores.

Parallelism is still very hard problem in theory, but its practical need isn't as prevalent as we thought on a decade-plus ago, partly thanks for the cloud and mobile. For most of us, at least the parallelism is kind of solved-by-someone-else problem. It is left for the small number of experts.

Concurrency is still there, but the situation is much better than before (async/await, immutable data types, actors...)

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#49

Post by dragontamer »

The free lunch isn't quite over, although significant advancements were made in parallel computing... it turns out that CPUs have been able to "auto-parallelize" your sequential code all along. Just not nearly as efficiently as explicitly parallel methodologies.

In 2005, your typical CPU was a 2.2GHz Athlon 64 3700+. In 2021, your typical CPU is a Ryzen 5700x at 3.8 GHz.

Single-threaded performance is far better than 72% faster however. The Ryzen 5700x has far more L3 cache, far more instructions-per-clock, far more execution resources than the ol' 2005 era Athlon.

In fact, server-class EPYC systems are commonly in the 2GHz range, because low-frequency saves a lot on power and servers want lower power usage. Today's EPYCs are still far faster per-core than the Athlons of old.

-------------

This is because your "single thread" is executed more-and-more in parallel today. Thanks to the magic of "dependency cutting" compilers, the compiler + CPU auto-parallelizes your code and runs them on the 8+ execution pipelines found on modern CPU "cores".

Traditionally, the CPUs in the 90s had a singular pipeline. But the 90s and 00s brought forth out of order execution, as well as parallel execution pipelines (aka: superscalar execution). That means 2 or more pipelines execute your "sequential" code, yes, in parallel. Modern cores have more than 8 pipelines and are more than capable of 4+ or 6+ operations per clock tick.

This is less efficient than explicit, programmer given parallelism. But it is far easier to accomplish. The Apple M1 continues this tradition of wider execution. I'm not sure if "sequential" is dead quite yet (even if there's a huge amount of machine working to auto-translate sequential into parallel technically... our code is largely written in a "sequential" fashion)

-------------

But the big advancements after 2005 was the rise of GPGPU compute. It was always known that SIMD (aka: GPUs) were the most parallel systems, from the late 1980s and early 90s the SIMD supercomputers always had the most FLOPs.

OpenCL and CUDA really took parallelism more / SIMD mainstream. And indeed: these SIMD systems (be it AMD MI100 or NVidia A100) are far more efficient and far higher compute capabilities than anything else.

The only "competitor" on the supercomputer scale is the Fugaku supercomputer, with SVE (512-bit SIMD) ARM using HBM RAM (same as the high-end GPUs). SIMD seems like the obvious parallel compute methodology if you really need tons and tons of compute power.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#50

Post by Const-me »

amelius wrote: post #46
Const-me wrote: post #42

Earlier quoted context omitted.

> on one machine I'm getting "GPU lost" error messages every once in a while during a long computation, and I have no other option than to reboot my machine. If that’s your computer, just change the registry setting disabling the TDR. By default, Windows uses 2 seconds to limit max.pipeline latency. When exceeded, the OS resets the GPU, restarts the driver, and logs that “device lost” in the event log. If that’s your…

Thanks for the advice (I'm on Linux), but my point is that the GPU really is not the "first-class citizen" that the CPU is.

> my point is that the GPU really is not the "first-class citizen" that the CPU is.

On Windows GPU is the first-class citizen since Vista. In Vista, Microsoft started to use D3D10 for their desktop compositor. In Windows 7 they have upgraded to Direct3D 11.

The transition wasn’t smooth. Only gamers had 3D GPUs before Vista, many people needed new computers. Technically, Microsoft had to change driver model to support a few required features.

On the bright side, now that XP->Win7 transition is long in the past, and 3D GPUs are used for everything on Windows. All web browsers are using D3D to render stuff, albeit not directly, through the higher-level libraries like Direct2D and DirectWrite.

Linux doesn’t even have these higher-level libraries. They are possible to implement on top of whichever GPU API is available https://github.com/Const-me/Vrmac#vector-graphics-engine but so far nobody did it well enough.

It’s very similar situation with GPU compute on Linux. The kernel and driver support has arrived by now, but the higher-level user mode things are still missing.

P.S. If you’re on Linux, pretty sure you can still refactor your compute shaders the same way and they will work fine afterwards. You obviously don’t have ID3D11Query/ID3D11Fence but most GPU APIs have replacements: VkFence(3) in Vulkan, EGL_KHR_fence_sync/GL_OES_EGL_sync/VG_KHR_EGL_sync extensions for GL and GLES, etc.

Post reply on HN