Live data from Hacker News

Ask HN: On Rob Pike's Concurrency is not Parallelism?

news.ycombinator.com

1–10 of 71 posts

Ask HN: On Rob Pike's Concurrency is not Parallelism?

#1
This is regarding slides by Rob Pike with above title. Every time I go thru this I feel like a moron. I'm not able to to figure out the gist of it. It's well understood that concurrency is decomposition of a complex problem into smaller components. If you cannot correctly divide something into smaller parts, it's hard to solve it using concurrency.

But there isn't much detail in slides on how to get parallelism once you've achieved concurrency. In the Lesson slide (num 52), he says Concurrency - "Maybe Even Parallel". But the question is - When and How can concurrency correctly and efficiently lead to Parallelism?

My guess is that, under the hood Rob's pointing out that developers should work at the level of concurrency - and parallelism should be language's/vm's concern (gomaxprocs?). Just care about intelligent decomposition into smaller units, concerned only about correct concurrency - parallelism will be take care by the "system".

Please shed some light.

Slides: http://concur.rspace.googlecode.com/hg/talk/concur.html#title-slide HN Discussion: http://news.ycombinator.com/item?id=3837147

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#2
I believe the concept says to focus more on task-based parallelism rather than data-based parallelism.

In Go, it is easy to create multiple tasks/workers each with a different job. This is implicitly parallelizable - each task can (but doesn't have to) work within their own thread. The only time when the workers can't run in parallel is when they are waiting on communication from another worker or outside process.

This is opposed to data level parallelism where each thread is doing the nearly exactly same instructions on different input, with little to no communication between the threads. An example would be to increase the blue level on each pixel in an image. Each pixel can be operated on individually and be performed in parallel.

So - the push is for more task-based parallelism in programs. It is very flexible in that it can run actually in parallel or sequentially and it won't matter on the outcome of the program.

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#3
I don't think he's saying that you shouldn't concern yourself at all with parallelism; only that you should focus on concurrency first and that will lead to easier parallelism. And he I think he is saying that decomposition and concurrency helps non-parallel programs stay simple and easy to understand. The benefits of concurrency are greater than just parallelism.

I think that is about what you are saying.

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#4
Parallelism is when you run your program on multiple processors. Semantics of your program does not change whether you run it on single processor or multiple processors.

Concurrency is when you write your program using multiple threads. Your program looks and means vastly different if you use threads.

You use concurrency not for performance gain, but for clarity of your program. You use parallelism for performance gain, to utilize all your processors.

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#5
IMO, they solve two different goals. Sometimes these goals overlap, but not always. Please someone correct me if I'm wrong, but this is how I see it:

The goal of concurrency is to model a problem that is easier or better or more natural to model concurrently (that is, different parts are running simultaneously). For example, if you are simulating agents in some virtual world (eg a game), then it may make sense that these agents are being modeled in a way that they are all running simultaneously and the processing of one does not block the processing another. This could be done by timeslicing available processing between each agent (either by using the processor/OS pre-emptive multitasking, if available, or through cooperative multitasking[1]), or is could be done by physically running multiple agents at the same time on different processors or cores or hardware threads (parallelism). The main point is that concurrency may be parallel, but does not have to be and the reason you want concurrency is because it is a good way to model the problem.

The goal of parallelism is to increase performance by running multiple bits of code in parallel, at the same time. Concurrency only calls for the illusion of parallelism, but parallelism calls for real actual multiple things running at the exact same time and so you must have multiple processors or cores or computers or whatever hardware resources for parallelism, while concurrency can be simulated on single core systems. Parallel code is concurrent code, but concurrent code is not necessarily parallel code.

Distributed programming is parallel programming where the code is running in parallel, but distributed over multiple computers (possibly over the internet at distant locations) instead of running locally on one multi-core machine or a HPC cluster.

From stackoverflow[2]:

    Quoting Sun's Multithreaded Programming Guide:
    
    Parallelism: A condition that arises when at least two threads are executing simultaneously.
    
    Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism.
As for when can concurrency be turned into parallelism, that depends. Assuming that the hardware resources exist (or that it simply falls back to time-sliced concurrency if they do not), parallelism can be achieved if there are multiple things that can execute independently. There are at least three types of parallelism and if your problem, code and/or data fit one of these, then your concurrent code may be executed in parallel.

1. You have a number of items of data and each one can be processed independently and at the same time. This is the classic embarrassingly parallel data parallelism. For example, you have a large array of input data and an algorithm that needs to run on each one, but they do not interact. Calculating pixel colours on your screen, or handling HTTP requests, for example.

2. You have two or more independent tasks doing different things that are run in parallel. For example, you have one thread handling the GUI and another thread handling audio. Both need to run at the same time, but both run independent of each other with minimal communication (which can happen over a queue, perhaps).

3. Sometimes you have a stream of data which must be processed by a number of tasks one after the other. Each task can be run in parallel so that if you have a stream of data, item[0], item[1], item[2], etc (where 0 is first in the stream, 1 is next and so on) and a number of tasks that need to run in order: A, B, C - then you can run A, B and C in parallel such that A processes item[0] while B and C are idle, then B processes item[0] an A processes item[1] and C is idle. Then C processes item[0] while B processes item[1] and A processes item[2] and so on. This is called pipelining and as you probably know is a very common technique inside processors.

Of course, all three can be combined.

[1] Could be a coroutine which is yielded or simply by executing some kind of update function which, by contract, must not block

[2] http://stackoverflow.com/questions/1050222/concurrency-vs-pa...

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#6
I think his point is that although people are often interested in parallel behavior, they should really focus more on concurrent design to avoid/remove the dependencies that ultimately limit parallelism. Slide 19 mentions automatic parallelization, but his point is that developers should think more about concurrency and not that Go will automatically maximally parallelize concurrent programs.

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#7
post #4

Parallelism is when you run your program on multiple processors. Semantics of your program does not change whether you run it on single processor or multiple processors. Concurrency is when you write your program using multiple threads. Your program looks and means vastly different if you use threads. You use concurrency not for performance gain, but for clarity of your program. You use parallelism for performance ga…

That can't be right. Threads don't improve the clarity of most programs; they're notoriously unclear.

Re: Ask HN: On Rob Pike's Concurrency is not Parallelism?

#9
post #2

I believe the concept says to focus more on task-based parallelism rather than data-based parallelism. In Go, it is easy to create multiple tasks/workers each with a different job. This is implicitly parallelizable - each task can (but doesn't have to) work within their own thread. The only time when the workers can't run in parallel is when they are waiting on communication from another worker or outside process. Th…

I don't agree with that; task parallelism is easier than data parallelism in an actor/CSP-based system, but both have their place. Take something like x264 -- task parallelism will not help it, unless you're encoding multiple videos at one time. But data parallelism (SIMD, in particular) is the reason it's the fastest encoder around.
Post reply on HN