Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The free lunch is over: a fundamental turn toward concurrency in software (2005)

gotw.ca

Post reply on HN
Print view View original

61–70 of 101 posts

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#61

Post by dragontamer »

vmchale wrote: post #58
snickerer wrote: post #56

I agree with this article a lot. The layers over layers of software bloat increased with the hardware speed. The general trade-off is: We make the software as slow as possible to have the shortest development time with the least educated programmers. "Hey web dev intern, click me a new UI in the next two hours! Hurry!" This intern-clicks-a-new-UI works because we have a dozen layers of incomprehension-supporting-tech…

> More parallel CPUs need higher skilled devs working slower to utilize the chip's full potential. Parallelism isn't so bad, you can use a functional style a lot of the time! E.g. Futhark, the accelerate framework, APLs....

Based on what I can tell... the programmers who made huge contributions to parallel compute are those programmers who grossly understood the hardware.

It didn't matter if they were Java programmers (Azul), Lisp Programmers (1980s CM2: Steel / Hille), GPU programmers, or C programmers. If you understood the machine, you figured out a fast and concurrent solution.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#62

Post by chrisseaton »

lostcolony wrote: post #60
chrisseaton wrote: post #59

Earlier quoted context omitted.

> Kind of strawmanning here, no? I don't think so? I'm responding to: > no programs needed changing at all to get the full advantage of all the cores (barring some special cases) Those 'special cases' are all the algorithms which aren't inherently parallel, which is most of them! How many algorithms inherently have a large amount of natural parallelism? I'd say it's very few. For example no matter how you write MD5 o…

As I said before - 'Obviously sequential tasks are run sequentially.' Innately sequential algorithms will still be sequential. Erlang already gave developers the tools, and incentive, to write things in a parallelized fashion. If a developer had a need that was directly sequential, it will, of course, still be sequential. If they had a need that would benefit from parallelism, they had the tools, the context, and the…

> they had the tools, the context, and the encouragement from the language itself to write it in parallel

No I think Erlang gives you the tools, context, and encouragement to write it concurrently. After you have concurrency, you may or may not have parallelism.

And if you don't you will have to restructure your program to get it, even if your program was already highly concurrent. That's contrary to what the person I was replying to said.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#64

Post by N1H1L »

Spivak wrote: post #28
typon wrote: post #15

I blame Python and Javascript. Two of the most popular languages without proper concurrency support.

What is Python missing in your eyes? Cooperative multitasking is a relatively recent thing to be in vogue again, and python has had good support for multithreading and multiprocessing since forever.

Also dask works pretty good all the way upto 200-250 CPU cores. It's pretty straightforward to build your code to use dask rather than numpy or pandas

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#65

Post by lostcolony »

chrisseaton wrote: post #62
lostcolony wrote: post #60

Earlier quoted context omitted.

As I said before - 'Obviously sequential tasks are run sequentially.' Innately sequential algorithms will still be sequential. Erlang already gave developers the tools, and incentive, to write things in a parallelized fashion. If a developer had a need that was directly sequential, it will, of course, still be sequential. If they had a need that would benefit from parallelism, they had the tools, the context, and the…

> they had the tools, the context, and the encouragement from the language itself to write it in parallel No I think Erlang gives you the tools, context, and encouragement to write it concurrently. After you have concurrency, you may or may not have parallelism. And if you don't you will have to restructure your program to get it, even if your program was already highly concurrent. That's contrary to what the person…

If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism?

I.e., simply spinning up an actor obviously does not imply concurrency; the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent one.

Likewise, if you have 5 processes all adding a list of numbers (independently and sequentially, one element at a time), but only 4 cores, you would have only 4 processes running in parallel at a time; a hardware limit. Up the cores, you can have all 5.

Can you give me an example of a concurrent program that is not also parallel, due to non-hardware reasons, in Erlang? Because switching the goalposts around - as you say, Erlang encourages you to write concurrent programs, even before CPUs allowed true parallelism, and because of that, once the CPU allowed true parallelism, it only required a VM tweak and suddenly Erlang programs were running in parallel. But you say no, that that wasn't the case; just because you wrote things to be concurrent, and now the hardware has changed to allow them to be in parallel, they aren't necessarily parallel. I'd love an example of that.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#66

Post by dTal »

chrisseaton wrote: post #59
lostcolony wrote: post #57

Earlier quoted context omitted.

Kind of strawmanning here, no? Obviously sequential tasks are run sequentially. The OP isn't saying "magic" happens. Just, Erlang was written from the beginning to encourage parallelized code. Its whole programming model is around that. Unlike pretty much every language it was contemporary with, which assumed that the default was a single unit of execution (and that code running in parallel was the exception). Think…

> Kind of strawmanning here, no? I don't think so? I'm responding to: > no programs needed changing at all to get the full advantage of all the cores (barring some special cases) Those 'special cases' are all the algorithms which aren't inherently parallel, which is most of them! How many algorithms inherently have a large amount of natural parallelism? I'd say it's very few. For example no matter how you write MD5 o…

Who says "most" algorithms aren't inherently parallel? How do you even enumerate that set? What counts as an "algorithm" anyway? Parallel processing is the norm in biology. And even if we grant that most of the algorithms humans have devised to run on our von Neumann machines are sequential - doesn't that say more about us, than about some intrinsic property of algorithms?

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#67

Post by chrisseaton »

lostcolony wrote: post #65
chrisseaton wrote: post #62

Earlier quoted context omitted.

> they had the tools, the context, and the encouragement from the language itself to write it in parallel No I think Erlang gives you the tools, context, and encouragement to write it concurrently. After you have concurrency, you may or may not have parallelism. And if you don't you will have to restructure your program to get it, even if your program was already highly concurrent. That's contrary to what the person…

If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? I.e., simply spinning up an actor obviously does not imply concurrency; the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent o…

> If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism?

You can have concurrent tasks but with dependencies between them which means there is no parallelism available.

> Can you give me an example of a concurrent program that is not also parallel, due to non-hardware reasons, in Erlang?

For example....

> the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent one.

No that is actually concurrent. These are two concurrent tasks. They just don't have any parallelism between them because there is a sequential dependency and only one is able to make progress at any given time.

Concurrent and parallel are not the same thing, if you didn't know. Concurrency is having work in progress on multiple tasks at the same time. You don't need to actually be advancing more than one task at the same time for it to be concurrent.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#68

Post by Jtsummers »

lostcolony wrote: post #65
chrisseaton wrote: post #62

Earlier quoted context omitted.

> they had the tools, the context, and the encouragement from the language itself to write it in parallel No I think Erlang gives you the tools, context, and encouragement to write it concurrently. After you have concurrency, you may or may not have parallelism. And if you don't you will have to restructure your program to get it, even if your program was already highly concurrent. That's contrary to what the person…

If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? I.e., simply spinning up an actor obviously does not imply concurrency; the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent o…

Concurrency produces parallelism when the concurrent portions can actually run at the same time. That is, between synchronization points which force sequential behavior.

It's very feasible to write a concurrent program that overuses (versus what is strictly needed) synchronization points in order to produce something which offers a better codebase (by some measure) but provides no parallelism. You have to remove/minimize these synchronization points in order to obtain parallelism. It's also possible that by design these synchronization points can't be eliminated.

Erlang's message passing is inherently asynchronous, but you could conceive of similar synchronization points in a program that will reduce the potential parallelism or eliminate it depending on the overall structure. For instance:

  PID ! {self(), Message}, ;; forgot self() initially
  receive
    {PID, Response} ->
  end,
  ...
You still have a concurrent design (which can be easier to reason about), but because you're waiting on a response you end up without any parallelism no matter how many cores you throw at it.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#69

Post by chrisseaton »

dTal wrote: post #66
chrisseaton wrote: post #59

Earlier quoted context omitted.

> Kind of strawmanning here, no? I don't think so? I'm responding to: > no programs needed changing at all to get the full advantage of all the cores (barring some special cases) Those 'special cases' are all the algorithms which aren't inherently parallel, which is most of them! How many algorithms inherently have a large amount of natural parallelism? I'd say it's very few. For example no matter how you write MD5 o…

Who says "most" algorithms aren't inherently parallel? How do you even enumerate that set? What counts as an "algorithm" anyway? Parallel processing is the norm in biology. And even if we grant that most of the algorithms humans have devised to run on our von Neumann machines are sequential - doesn't that say more about us, than about some intrinsic property of algorithms?

I mean just take a look in any book of algorithms and think for yourself how many have inherent parallelism.

Re: The free lunch is over: a fundamental turn toward concurrency in software (2005)

#70

Post by kaliszad »

gnarbarian wrote: post #40
maattdd wrote: post #16

Earlier quoted context omitted.

You mean parallelism ? JS has good concurrency support (event, async..)

Parallelism in JavaScript is simple using web workers. of course the tricky part as with all parallel applications is managing the complexity you create yourself. The only languages that seem to do a good job at handling this for you are challenging in other ways like Haskell and Erlang.

I don't think web workers are a good design. They are maybe simple, but lack in flexibility and capability. E.g. if I am not mistaken, you cannot control animations from a web worker. You also have to communicate with them using strings, therefore there is considerable overhead.

JavaScript and the related ecosystem in the browser lack a number of things, that are then patched over using additional complexity such as web assembly, bug handling/ feature workarounds in applications, mobile apps that you basically have to develop, because the web isn't useable/ performant enough for some stuff. Of course we have some extra experience now and it is easy to criticize in hindsight. E.g. the decision to make JavaScript less LISPy was perhaps a good decision for it's large adoption but longer term the homoiconicity could have been used for easier generation instead of e.g. stuff like WebAssembly. We also know, that stuff like Clojure(Script) is possible now. We also know, that we really want a native 64bit integer type instead of hodge-podge solutions with reduced precision of 53 bits or big decimal with worse/ complicated performance characteristics.

For the last 12 or so years, the web and related technologies became an application platform and a video delivery network with demands that rival native applications. We even use JavaScript on the server side through Node.JS. This has enabled tremendous things to happen but it is also perhaps the least stable platform to develop for. The web actively breaks some of the more complex applications that it has enabled precisely because the foundations are not solid enough. The current state of affairs is we keep piling on new and shiny APIs and technologies, that perhaps have considerable merit but we don't really fix the broken things in a way normal devs could keep up. I mean, how do you imagine to keep up, if even GMail, Google Maps, YouTube, Facebook and all its web apps, even major news sites and other applications backed by big companies still have rather obvious bugs in these products? I guess, "move fast and break things [for the later generations to fix all this mess]" it is.

Post reply on HN