61–70 of 88 posts
You can track development here: https://github.com/jasonswearingen/godot-csharp-tech
I'm just starting the implementation, but I've done a similar system before so pretty sure I'm not in over my head :)
Earlier quoted context omitted.
I have to second that. Parallel programming is extremely hard to get right and people persistently underestimate it. For example, I've seen many times code that erroneously assumed that values can be read from global memory concurrently if only write access is guarded. Of course, it depends a lot on what your programming language has to offer. For example,in my experience Ada makes it easier to get parallelism correc…
> For example, I've seen many times code that erroneously assumed that values can be read from global memory concurrently if only write access is guarded. As someone who has made that assumption, what's wrong with it?
When you are missing the synchronization chain, the issue you can run into is that the value of other memory locations is undefined. The sequence of accesses to any given location does comprise a total order (due to cache coherence), but it is not generally the case that the ordering of accesses to different locations will be agreed on by different threads.
Earlier quoted context omitted.
Still easy to mess up the coordination when you have mutable shared state.
Embarrassingly parallel workloads kinda don't have mutable shared state... that's why they're "embarrassing".
You still need to create all the parallel threads/processes/fibers/whatever, divide the work between them and collect or merge the results.
And if you work in an environment where state is mutable and shared per default (i.e. your typical shared-memory multithreaded Java/C#/C++/Ruby program), then it's still very easy to mess up somewhere.
Understand the fundamental problems/principles. Almost all problems and solutions stem from one limitation: only one writer at any given time. Immutable means one writer. Locking means one writer. Splitting a list and processing each half means one writer. The actor model means one writer. All are valid solutions to your problems. A piece of personal advice: Avoid acquiring more than one lock at the same time if poss…
Understand the fundamental problems/principles. Almost all problems and solutions stem from one limitation: only one writer at any given time. Immutable means one writer. Locking means one writer. Splitting a list and processing each half means one writer. The actor model means one writer. All are valid solutions to your problems. A piece of personal advice: Avoid acquiring more than one lock at the same time if poss…
Parallel programming in Python is hard. On Windows, it's hell. In Matlab it's a breeze. Just put "par" in front of the thing. Done. If you wanna shell out money to enable that functionality, that is.
Once a pointer points at something in your parallel task, software is unable to reason about it, erring on the side of caution and thus, parallelism wont happen there.
Also these parallelisms, usually work for trivial parallel structures like for loops. Easy to unroll into multiple cores, but also with little gain, as communication times eats the gained advantages.
Understand the fundamental problems/principles. Almost all problems and solutions stem from one limitation: only one writer at any given time. Immutable means one writer. Locking means one writer. Splitting a list and processing each half means one writer. The actor model means one writer. All are valid solutions to your problems. A piece of personal advice: Avoid acquiring more than one lock at the same time if poss…
Of course this only work if all locks you need to take are exposed and not hidden behind some abstraction. Then again, when they are it is hard to make sure you only take one lock at a time anyway...
Understand the fundamental problems/principles. Almost all problems and solutions stem from one limitation: only one writer at any given time. Immutable means one writer. Locking means one writer. Splitting a list and processing each half means one writer. The actor model means one writer. All are valid solutions to your problems. A piece of personal advice: Avoid acquiring more than one lock at the same time if poss…
Queues are my advice. The locking is all in the queue operations. Everybody else just takes and posts work items to the queue. Can even use lockless queues.
Earlier quoted context omitted.
This gives me two questions: - did you have a method/tool to debug (from book, or chatting with others) - are there still cpu designs dedicated to parallel workloads (I assume the usual desktop intel/amd, no matter how brilliant, may not be without hiccups in high parallelism) oh one last thing, do you use dedicated compilers for parallelization ? or is it something "mainstream" like openmp
I can greatly recommended mozilla's rr for hunting concurrency bugs, as it allows efficient replay and provides a scheduler that is designed to increase the probability of triggering concurrency bugs. The issue with x86 is simple: the memory model pretty much requires loads of broadcasts on the SMP interconnect. IIRC Power9 is nice due to (1) a memory model that doesn't require this (much) broadcasting on the interco…
and
2) I can't find any hit about power9 remote atomics outisde of GPU memory. Any pointers?
Earlier quoted context omitted.
Queues are my advice. The locking is all in the queue operations. Everybody else just takes and posts work items to the queue. Can even use lockless queues.
how would you implement, say, a concurrent hash map with queues?
But more likely a hashmap is too fine a granularity for queues. If a hashmap can be owned by one process, that would perform better.