Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is parallel programming hard, and, if so, what can you do about it?

mirrors.edge.kernel.org

Post reply on HN
Print view View original

151–160 of 199 posts

Re: Is parallel programming hard, and, if so, what can you do about it?

#151

Post by CyberDildonics »

vore wrote: post #110
crabbone wrote: post #106

Earlier quoted context omitted.

I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was released. So, I associate this nonsense with Go, but it could have happened earlier, I simply never heard about it before then. Anyways. The way I see it used today, it's applied to language runtimes incapable or severely crippled when it comes to parallel / conc…

How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.

Parallelism is a physical thing, concurrency is a logical thing.

Fundamentally the difficulty is all about synchronization. People can try to split hairs and say there are two terms for two different things but ultimately it doesn't matter because the underlying problem is the same.

Re: Is parallel programming hard, and, if so, what can you do about it?

#152

Post by OkayPhysicist »

cpach wrote: post #50
PheonixPharts wrote: post #30

Earlier quoted context omitted.

There are multiple replies like this one, but it's a bit shocking to see that on Hacker News people don't know the difference between concurrent programming and parallel programming. Concurrency means that you can have multiple tasks running in the same time period, Parallelism means you have multiple tasks running at the same time . The most obvious demonstration of this is that you can (and many languages do) have…

Correct me if I’m wrong, but isn’t concurrency enough for “most” use-cases? When does one really need true parallelism?

Concurrenceny deals with what can be parallel, because they don't share data dependencies. For example, a map() function is trivially a statement of concurrency: each item needs to have a function applied to it, and assuming pure functions, that can all be done concurrently.

Parallel computing cares about what should be parallel, i.e., actually implementing parallelism. For most programmers, this job can (and probably should) be left to the scheduler, whose job is to translate concurrency into reasonably sane parallelism.

The scheduler comes with overhead, though, that can be avoided with hand-spun parallelism. I like to think about it like manual memory management: for most programs and programmers, using a garbage collector that a memory management expert wrote is easier/better than manually allocating and freeing memory, but there are performance gains to be had if you don't.

Re: Is parallel programming hard, and, if so, what can you do about it?

#153

Post by preseinger »

crabbone wrote: post #141
vore wrote: post #110

Earlier quoted context omitted.

How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.

You have just repeated the nonsense I was talking about. The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of deali…

a program (as written) is either concurrent or not, a program (as executed) is either parallel or not

a program which is not written as concurrent can never be executed as parallel

a program which is written as concurrent can be executed as parallel, or not

> The situation you describe (when there's a single processor core) isn't parallel or concurrent.

a program running on a single core can never be parallel, but it can be concurrent

concurrent is a logical property, parallel is a physical property

Re: Is parallel programming hard, and, if so, what can you do about it?

#154

Post by mmcnl »

crabbone wrote: post #106
ascar wrote: post #84

I see a lot of confusion between parallel programming [1] and concurrent programming [2] in the comments here. The former and what this book is about deals with the problem of parallelizaing a single sequential program. There usually is strong interaction or dependencies between elements and progress needs synchronization. E.g. timestep iterations in real-time simulations that need synchronization with data communica…

I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was released. So, I associate this nonsense with Go, but it could have happened earlier, I simply never heard about it before then. Anyways. The way I see it used today, it's applied to language runtimes incapable or severely crippled when it comes to parallel / conc…

Just because you didn't know about the concept doesn't mean the distinction is nonsense. I think they are similar but not the same, exactly for the reasons laid out in the comment you are replying too. Just because you hate the languages that support concurrent programming doesn't mean concurrent programming is meaningless. Any language using async/await (basically all of them these days) support concurrent programming, including languages such as Swift and C# which are nothing like Python or JavaScript.

Re: Is parallel programming hard, and, if so, what can you do about it?

#155

Post by preseinger »

crabbone wrote: post #147
titzer wrote: post #136

Earlier quoted context omitted.

Parallelism and concurrency are different models. It is not a needless distinction. With just concurrency, you can have a system, e.g. an event processing system, where each event happens, is processed to completion, and then the next event is processed. Events can be of different types and can have different handlers. Events can be arrive (or be delivered) in different orders. The context-switching points are known…

This is word salad. Sorry. What are you modeling? The way you use "concurrency" it's indistinguishable from code without any signs of parallelism. From your "definition", concurrency is just any code. Such definitions are called "trivial" if you don't want to offend the author, and "worthless" if you are honest. Your attempt at defining "parallelism" is even worse... You start by calling it undefined, and then procee…

a program which is concurrent expresses logically independent paths of execution, which can be executed one after the other, or at the same time, or any mix thereof -- it is a logical property of the program as expressed, and not necessarily exploited, or observable, when the program is run

a program which is parallel actively demonstrates multiple paths of execution during runtime -- it is a physical property of the program as executed

tl;dr: every parallel program is concurrent, not every concurrent program is parallel

Re: Is parallel programming hard, and, if so, what can you do about it?

#156

Post by preseinger »

aleph_minus_one wrote: post #39
vmfunction wrote: post #32

Earlier quoted context omitted.

what about "green threads" that is not managed by the OS like https://tokio.rs ?

Green threads do not make use of multiple cores of a modern processor.

what makes you say this?

go programs definitely saturate modern server-class CPUs

Re: Is parallel programming hard, and, if so, what can you do about it?

#157

Post by CyberDildonics »

spacechild1 wrote: post #85
Retric wrote: post #71

Earlier quoted context omitted.

> What problems exist that generate events or commands faster than 500 million per second? AAA games, Google search, Weather simulation, etc? I mean it depends on what level of granularity you’re talking about, but many problems have a great deal going on under the hood and need to be multi threaded.

Don't forget about audio!

I don't think audio applies here at all.

Each channel can be mixed in parallel, 44,100 samples per second is not much per channel and mixing isn't difficult.

Also most can be cached and don't need to be mixed or filtered in real time because they haven't been changed from the last play.

Re: Is parallel programming hard, and, if so, what can you do about it?

#158

Post by vore »

CyberDildonics wrote: post #151
vore wrote: post #110

Earlier quoted context omitted.

How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.

Parallelism is a physical thing, concurrency is a logical thing. Fundamentally the difficulty is all about synchronization. People can try to split hairs and say there are two terms for two different things but ultimately it doesn't matter because the underlying problem is the same.

Single core concurrency doesn't have to deal with hardware memory synchronization.

Re: Is parallel programming hard, and, if so, what can you do about it?

#159

Post by OkayPhysicist »

kllrnohj wrote: post #43
vmfunction wrote: post #32

Earlier quoted context omitted.

what about "green threads" that is not managed by the OS like https://tokio.rs ?

You only ever need this if you're trying to have hundreds of thousands to millions of threads. It's a very niche problem to have

It's a niche problem to have because our current programming paradigm treats concurrency as a second-class citizen. The invariants of most languages do not include those required for mass concurrency.

Functional programming better aligns with the requirements, which is how you arrive at Erlang and Elixir. Every map() function can be trivially replaced with the concurrent cmap(), because the side effects that would make it non-trivial are impossible to express.

Given that Moore's law as it applied to single threaded performance is dead and in the ground, it makes a lot of sense to start paying more attention to systems that treat concurrency as more than an afterthought.

Re: Is parallel programming hard, and, if so, what can you do about it?

#160

Post by titzer »

crabbone wrote: post #147
titzer wrote: post #136

Earlier quoted context omitted.

Parallelism and concurrency are different models. It is not a needless distinction. With just concurrency, you can have a system, e.g. an event processing system, where each event happens, is processed to completion, and then the next event is processed. Events can be of different types and can have different handlers. Events can be arrive (or be delivered) in different orders. The context-switching points are known…

This is word salad. Sorry. What are you modeling? The way you use "concurrency" it's indistinguishable from code without any signs of parallelism. From your "definition", concurrency is just any code. Such definitions are called "trivial" if you don't want to offend the author, and "worthless" if you are honest. Your attempt at defining "parallelism" is even worse... You start by calling it undefined, and then procee…

Your comment is subtractive from the discussion and its tone and attitude is not OK. If you came here to make no attempt to understand what people have written, even when they are explaining words, defining terms, and instead attack them and shout past them, we're better off if you didn't.

I wouldn't normally put such a fine point on it, but in this thread you've managed to make a lot of noise and calls lots of things "nonsense". If you're after emotional responses and heated discussion, that's tantamount to trolling, and you should stop.

Post reply on HN