I have almost finished the book (to be honest, I have skipped the "Validation" chapter) and I found it quite interesting. In particular I liked that it also covered more "exotic" topics such as sequence locks, hazard pointers and RCU. However, in general, I found it a bit too much focussed on Linux kernel code. The code examples are full of kernel code macros, such as READ_ONCE and WRITE_ONCE, instead of C11/C++11 atomics. In the end, I appreciated it because it gave me a new perspective, but I'm not sure if this helpful for people who are not already familiar with parallel programming...
Is parallel programming hard, and, if so, what can you do about it?
181–190 of 199 posts
Re: Is parallel programming hard, and, if so, what can you do about it?
#182Earlier quoted context omitted.
>> don't know who invented this nonsense distinction ... It not nonsense. In C or C++ lots of code can be made parallel using OpenMP and inserting some #pragma statements above for loops. This does not work for things like running a UI in one thread and some other work in another thread, perhaps displaying results as they are found. These are quite different types of parallelism.
I think it's nonsense to invent new terminology for this tiny minor difference in how you use threads and whether you wait on results or you don't wait.
Concurrency didn’t necessarily require threads. Concurrency is not a difference in how you use threads, its logical computer science concept. Parallelism is a physical property and parallelism is almost always concerned with doing things a faster by doing them in parallel, concurrency can be done for other purposes (eg to not have long running tasks block work). All parallel programming is concurrent but not all concurrent programming is parallel.
The distinction matters because the focus is different: they’re different topics. With concurrent programming you deal with synchronisation and topics like lock-freedom and wait-freedom. In parallel programming the main focus is about work distribution and scheduling to process it faster.
Re: Is parallel programming hard, and, if so, what can you do about it?
#183Earlier quoted context omitted.
Yes? Parallelism is inherently concurrent but concurrency is not inherently parallel. But single core concurrency still exists as a form of non-parallel concurrency. The point I'm trying to make is down to brass tacks, single core concurrency is a lot simpler than multi core concurrency, because you don't need any hardware cooperation.
Before you were saying they are two different things, now you're saying they are the same when you have multiple cores and not the same when you have a single core. If they're the same when you have multiple cores, isn't there just single and multi-core concurrency by your own definitions? If so, why are you making a distinction of parallelism and concurrency?
Parallelism is concurrent.
Concurrency isn’t necessarily parallel.
There are still plenty of single core systems out there running concurrent software, even today, for example embedded systems.
Re: Is parallel programming hard, and, if so, what can you do about it?
#184Earlier quoted context omitted.
but it's just not a tiny minor difference. The problems you end up dealing with are entirely different. The terminology used might be terrible since they're originally synonyms. Differentiating between the two is very useful though.
how are the problems different? In all cases you want the maximum performance. The only decision here is how you schedule your tasks. Ideally entirely in parallel and you only schedule them to wait if you can't find a way to make them 100% parallel
> In all cases you want the maximum performance.
Not necessarily true. In many cases of concurrent programming, you don’t actually care about performance, you care about multiple different things happening seemingly at the same time (sure faster is nice, but not stuttering or freezing is more important). For example on a single core multitasking system (either in the past or on an embedded system today), you want multiple different tasks all running together, but this is achieved by giving them all a little chunk of execution one after the other — time slicing. The tasks are running concurrently, they all make progress together, but they share the same single thread of execution, they don’t happen in parallel. Yet one doesn’t block another, which is the important thing here.
I mean may e you “want” maximum performance, but that’s not what’s most important. Not having one task block another is what’s important.
Parallel programming is pretty much always about making things faster, by running then in parallel at the same time. The main benefit of doing that is that they both complete faster.
Re: Is parallel programming hard, and, if so, what can you do about it?
#185[dead]
Re: Is parallel programming hard, and, if so, what can you do about it?
#186Earlier quoted context omitted.
How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.
You have just repeated the nonsense I was talking about. The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of deali…
Your "single core" computer contains a lot of other concurrently operating hardware like storage and network cards.
Re: Is parallel programming hard, and, if so, what can you do about it?
#187Earlier quoted context omitted.
GCD/libdispatch is a fantastic approach to concurrency and you can build and install support for non-Apple operating systems: https://github.com/apple/swift-corelibs-libdispatch Here’s a simple echo server: https://github.com/williamcotton/c_playground/blob/master/sr... Here’s a simple multithreaded database pool: https://github.com/williamcotton/express-c/blob/master/src/d...
libdispatch idea of using specific queues for serialized concurrency is nice, but its abstractions on top are unfortunately not as efficiently designed as it could be; `dispatch_source` doesn't allow for direct completion based IO schemes (io_uring, IOCP), pushing to a `dispatch_queue` always requires a heap allocation, `dispatch_semaphore/dispatch_sync` blocks the thread instead of yielding asychronously (can cause…
> `dispatch_semaphore/dispatch_sync` blocks the thread instead of yielding asychronously (can cause "thread explosion")
dispatch_group can do waits without blocking threads, but asynchronicity is overrated, and dispatch's design makes it easy to overdo. Having default global concurrent queues was probably a mistake.
Swift concurrency is a more modern design here.
Re: Is parallel programming hard, and, if so, what can you do about it?
#188Earlier quoted context omitted.
Thanks, skimmed a bit of the first half but will get into it more after my work day.
So, going through this a bit more on lunch break; I think the root of some of the disillusionment faced here -- and probably the lack of success in STM at this level generally -- is that they tried to do too much. I don't necessarily want the whole memory model of the runtime or VM to offer transactions necessarily. What I think is a good idea is to offer an overall framework -- within which higher level applications…
> Yes, users could escape it easily, and start doing inconsistent things. But that's on them, same as any other framework.
We're already there! You don't need an STM framework in order to leave consistency up to the user.
Re: Is parallel programming hard, and, if so, what can you do about it?
#189Earlier quoted context omitted.
You have just repeated the nonsense I was talking about. The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of deali…
Yes, concurrency on a single core CPU is simply not possible. That's why multitasking OS's didn't exist until multicore CPUs became a thing. If only these "Gophers" knew anything about computing history!
So before 2001 your OS didn't have a scheduler and was incapable of handling more than one process at the same time? The user interface was simply hanging when you told your CPU to do something?
Re: Is parallel programming hard, and, if so, what can you do about it?
#190I’m way-above-average interested in concurrent programming but this 600+ page brick will probably remain on my reading list until I am stranded on a deserted island. Did anyone here read the whole thing? Can one make a reasonable summary or is this more of a lexicon of different techniques?
I was going to read it, but then i played a round of Qwitzatteracht, the golf game, instead.