Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Joe Armstrong: Solving the wrong problem

joearms.github.com

Post reply on HN
Print view View original

141–150 of 169 posts

Re: Joe Armstrong: Solving the wrong problem

#141

Post by surferbayarea »

coldtea wrote: post #126
surferbayarea wrote: post #34

Have you even used zlib in c++? The largest ecommerce site out there uses zlib in a multitreaded c++ application(24 cores, 100s of threads, 1000s requests/sec/server) and it works just fine! Bet you erlang can't come within a tenth of the performance of c++...

> 1000s requests/sec/server That's not particularly impressive you know.

yes, because it also does other computation. Poiint was to illustrate that zlib can be used in a concurrent computing setting with high performance. The blog writer had claimed that zlib doesn't work in a multithreaded setting.

Re: Joe Armstrong: Solving the wrong problem

#142

Post by reeses »

seanmcdirmid wrote: post #120
willvarfar wrote: post #113

Earlier quoted context omitted.

Isn't Joe's post specifically about parallelism and how Erlang is designed for it?

The post is about concurrency, but the word parallelism is used instead. To be fair, task level parallelism makes concurrent code run faster, but doesn't really scale if done for its own sake.

I think in this case they're relatively interchangeable terms. Rather than a SIMD vectorization of a task, you are applying a MIMD solution to various parts of a task.

You can typically get more of an immediate boost with SIMD on current hardware (especially if you can effectively cast it to GPGPUs), but MIMD is more easily applied. Almost any application can be refactored to spawn lightweight threads for many calculations without any explicit knowledge of the π-calculus.

To your point and for a well-understood example, make -j doesn't always result in faster compilations. It may if you have the ability to leverage imbalances in CPU and storage hierarchies, but you can also kill your performance with context switches (including disk seeking).

Re: Joe Armstrong: Solving the wrong problem

#143

Post by SoftwareMaven »

reeses wrote: post #136
SoftwareMaven wrote: post #78

Earlier quoted context omitted.

How would you get per-actor heaps that cannot be violated by other actors? That is critical to Erlang's ability to recover from processes dying. I spent a lot of time doing Java and can't think how you could (you could in the JVM if you had language constructs for it, but then we are back to a new language). There's a reason Stackless Python's actors aren't just a library on top of Python.

TLABs are a partial step in that direction at the JVM level. You could also use object pools/factories keyed to the thread. Those are the first two "ghetto" hack solutions I can think of that wouldn't require significant code changes on a going-forward basis.

But those hacks wouldn't provide the same guarantees that language-level changes provide. Sure, you can try not to impact other thread's heaps, but nothing is stopping me, which means a simple programming error has the potential to impact multiple threads. As a result, you can't just "reboot" that thread (a critical piece of what makes Erlang interesting), because you have no guarantees its errors didn't impact other threads. You also have no guarantees that the underlying libraries aren't mucking up all of your carefully crafted memory management.

It's like the kernel protecting memory so applications can't overwrite each other. Sure, applications could just write to their own memory, but nobody actually trusts that model[1]. Instead, they want something below that level enforcing good behavior.

1. Obviously, virtual memory adds a wrinkle to this that kind of forces kernel protection, but even if we had literally unlimited RAM, we would still implement kernel protections on memory.

Re: Joe Armstrong: Solving the wrong problem

#144

Post by reeses »

seanmcdirmid wrote: post #89
unoti wrote: post #86

Earlier quoted context omitted.

Isn't it really fair to say that it's designed for both? The way it uses immutable state and something-similar-to-s-expressions to express data make it very straightforward (or even transparent) to distribute work between multiple processes and separate computers, in addition to how it makes it practical and simple to break work into small chunks that can be interleaved easily within the same thread. It's really desi…

Not at all. Erlang isn't useful for modern parallel computing as we know it, which is usually done as some kind of SIMD program; say MapReduce or GPGPU using something like CUDA. The benefit doesn't just come from operating on data all at once, but these systems (or the programmer) also do a lot of work to optimize the I/O and cache characteristics of the computation. Actor architectures are only useful for task para…

SIMD is a specialized form of parallelism. It is not the only definition of the term.

It should also be clear that task parallelism (or concurrency from your perspective) has not had the benefit of billions of engineer-hours focused on improving its performance. It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output. This would present, for example, in a proportional percentage of video frames with conspicuously different coloration.

Re: Joe Armstrong: Solving the wrong problem

#145

Post by reeses »

SoftwareMaven wrote: post #143
reeses wrote: post #136

Earlier quoted context omitted.

TLABs are a partial step in that direction at the JVM level. You could also use object pools/factories keyed to the thread. Those are the first two "ghetto" hack solutions I can think of that wouldn't require significant code changes on a going-forward basis.

But those hacks wouldn't provide the same guarantees that language-level changes provide. Sure, you can try not to impact other thread's heaps, but nothing is stopping me, which means a simple programming error has the potential to impact multiple threads. As a result, you can't just "reboot" that thread (a critical piece of what makes Erlang interesting), because you have no guarantees its errors didn't impact other…

Unfortunately, with any native code interaction, whether explicit FFI/NIF or implicit (RNG, clock, etc.), your threads can still mingle state.

In a single process, you're still limited by the memory virtualization capabilities of your underlying processor and MMU.

Re: Joe Armstrong: Solving the wrong problem

#146

Post by reeses »

Shish2k wrote: post #127
abc_lisper wrote: post #38

Earlier quoted context omitted.

Now, I have never read or written PHP. Here's the thing. PHP fills a niche. That niche is being super-productive early on. There are other goals, like, the code being readable, ease of being reasoned about, fast and so on. And there are languages that fill those niches. When people say crappy language, more often than not, crappy for their needs(or sometimes their expertise). There is no need to fight about it; its l…

Personally I call PHP crappy because it could easily have been much better at its main goal without sacrificing anything from its other goals, but for some reason it wasn't, and now we have to live with the stupid early decisions for compatibility reasons (eg there's no good reason for the mess of strFunc, str_func, string_func, funcstr, needle/haystack, haystack/needle, etc. When the main goal is being super-product…

Rasmus has clearly stated that PHP's ugliness is a feature. http://osdir.com/ml/php.devel/2002-10/msg00704.html

He freely admits that he is a crappy programmer who can't write languages. His goal was to re-use code because he hated (hates) programming.

That doesn't really explain why contributors perpetuate the atrocity, but understand that PHP is the way it is based on a set of priorities that many of us do not value.

Re: Joe Armstrong: Solving the wrong problem

#147

Post by willvarfar »

seanmcdirmid wrote: post #124
willvarfar wrote: post #122

Earlier quoted context omitted.

I think the post is about parallelism. Its about how Erlang naturally scales to many cores by running in parallel. If Erlang had only concurrency, as Javascript does, it would not be solving the "right problem".

Again, even if Erlang supports hardware threading properly, it doesn't magically become a good platform for parallel computing, there is no guarantee it will scale at all. Its funny actually: a PL person thinks the key to scalable parallelism is hardware threading; a graphics or systems person thinks the key is a well planned pipeline.

http://talks.golang.org/2012/waza.slide

"Once we have the breakdown, parallelization can fall out and correctness is easy."

Joe is saying this too. And he's saying that because Erlang is a concurrent language, parallelism (he's thinking MIMD not SIMD) is easy. He says:

> Now Erlang is (in case you missed it) a concurrent language, so Erlang programs should in principle go a lot faster when run on parallel computers, the only thing that stops this is if the Erlang programs have sequential bottlenecks.

I don't think he - nor the Go chaps - conflate concurrency and parallelism.

Re: Joe Armstrong: Solving the wrong problem

#148

Post by seanmcdirmid »

willvarfar wrote: post #147
seanmcdirmid wrote: post #124

Earlier quoted context omitted.

Again, even if Erlang supports hardware threading properly, it doesn't magically become a good platform for parallel computing, there is no guarantee it will scale at all. Its funny actually: a PL person thinks the key to scalable parallelism is hardware threading; a graphics or systems person thinks the key is a well planned pipeline.

http://talks.golang.org/2012/waza.slide "Once we have the breakdown, parallelization can fall out and correctness is easy." Joe is saying this too. And he's saying that because Erlang is a concurrent language, parallelism (he's thinking MIMD not SIMD) is easy. He says: > Now Erlang is (in case you missed it) a concurrent language, so Erlang programs should in principle go a lot faster when run on parallel computers,…

My main issue here is that people here "parallelism" and "a lot faster" they automatically think "scaling." But just hardware threading doesn't get us anywhere near that goal, even if we write our C-style multi-threaded code by hand very carefully.

The PL community is still not having honest up-to-date conversations about parallelism; they are about 20 years behind other fields.

Re: Joe Armstrong: Solving the wrong problem

#149

Post by seanmcdirmid »

reeses wrote: post #144
seanmcdirmid wrote: post #89

Earlier quoted context omitted.

Not at all. Erlang isn't useful for modern parallel computing as we know it, which is usually done as some kind of SIMD program; say MapReduce or GPGPU using something like CUDA. The benefit doesn't just come from operating on data all at once, but these systems (or the programmer) also do a lot of work to optimize the I/O and cache characteristics of the computation. Actor architectures are only useful for task para…

SIMD is a specialized form of parallelism. It is not the only definition of the term. It should also be clear that task parallelism (or concurrency from your perspective) has not had the benefit of billions of engineer-hours focused on improving its performance. It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized…

Task parallelism was something used to work on 20 years ago when we thought it was the solution to scaling. But then we found that the supercomputer people were right all along, that the only thing that really scales very well is data parallelism. So the focus in the last 5/10 years has been finding data parallel solutions to the problems we care about (say deep neural network training), and then mapping them to either a distributed pipeline (MapReduce) or GPU solution.

> It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

You are still thinking about concurrency, not parallelism. Yes, the cluster people had to think this way, they were interested in performance for processing many jobs; no the HPC people who needed performance never thought like this, they were only interested in the performance of one job.

> As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output.

Part of the problem, I think, is that we've been confused for a long time. Our PHBs saw problems (say massive video frame processing) and saw solutions that were completely inappropriate for it (cluster computing). Its only recently that we've realized there are often other/better option (like running MapReduce on that cluster).

Re: Joe Armstrong: Solving the wrong problem

#150

Post by seanmcdirmid »

reeses wrote: post #142
seanmcdirmid wrote: post #120

Earlier quoted context omitted.

The post is about concurrency, but the word parallelism is used instead. To be fair, task level parallelism makes concurrent code run faster, but doesn't really scale if done for its own sake.

I think in this case they're relatively interchangeable terms. Rather than a SIMD vectorization of a task, you are applying a MIMD solution to various parts of a task. You can typically get more of an immediate boost with SIMD on current hardware (especially if you can effectively cast it to GPGPUs), but MIMD is more easily applied. Almost any application can be refactored to spawn lightweight threads for many calcul…

> but MIMD is more easily applied. Almost any application can be refactored to spawn lightweight threads for many calculations without any explicit knowledge of the π-calculus.

MIMD hasn't been shown to scale, and its not locking that is the problem, but I/O and memory.

> To your point and for a well-understood example, make -j doesn't always result in faster compilations. It may if you have the ability to leverage imbalances in CPU and storage hierarchies, but you can also kill your performance with context switches (including disk seeking).

When Martin Odersky began pushing actors as a solution to Scala and multi-core, this was my immediate thought: the Scala compiler is slow, can this make it go faster? It was pretty obvious after some thought that the answer was no. But then we have no way yet of casting compilation as a data-parallel task (a point in favor of task parallelism, but it doesn't help us much).

Post reply on HN