Why Erlang Matters

    for(;;){
    }

(2) Erlang VM instance (it is called a node) is an OS process (plus some helper processes, but they are not important in this case) is of course one heap from kernel's point of view. But internal allocator keep spaces separated for BEAM's data. It is not always as basic as in some cases for binary blocks and sub-blocks it can actually share and ref-count them.

In the new release I like that it could have mailbox outside the main heap of each Erlang process. That could be an interesting parameter to play with.

As others have indicated, they're extremely lightweight, cheap to create and throw away, and are load balanced across cores.

But also, each has isolated memory, with share nothing semantics (with a couple caveats) which means that an exception in one won't affect others -unless you want it to-. That's huge.

But as has also been mentioned, you can create many of them. Someone else threw out 50k; nevermind that, try a million of them on a single box. That kind of concurrency opens up an entirely new paradigm of coding. One that is actually very useful, because it turns out, a lot of problems are naturally concurrent problems, that we've been trained to think about in sequential patterns because of how hard concurrency is.

An example I like to give is from the real world - task scheduling. We had to write some simple task scheduling for an application. Each task was multiple steps, many of which were time based (i.e., "execute this command, wait X amount of time, execute another command, wait Y amount of time, execute a third command, once that is successful execute a fourth command"). The traditional way of doing this would be some sort of priority queue, with tasks weighted by how long from now until they were to be done. You check how long until the next event, sleep until then, fire it, then repeat. Simple, right?

Except...each event leads to more events. And event timings can change. And events can happen simultaneously, so you actually need a pool of threads to actually execute the events on. Locks everywhere. Task logic is very hard to isolate from the execution logic (i.e., the bit that says "do X, and create an event to execute at time Y" is hard to keep entirely separate from the "pull event from priority queue, throw to a new thread to execute on, sleep until next event", since there are so many interactions between the two that can affect one another, changing when events happen, and when the queue puller needs to wake up).

In Erlang though? Trivial. Write your entire task as a single job. I.e., do x, wait, do y, wait, do z, wait for a message that z has completed, do a, etc. Then spin up one of those for each task that you need and let the VM handle the concurrency aspects of it. Even additional complexity, like "in the event of a message, change the amount of time until the next event to be half of what it was" is trivial; it's all contained in the same module, it all describes the same lifecycle of a single task. All the concurrency, the running of many of those tasks, and their interactions, and ensuring none is blocked, etc, is -free-.

This sounds like an obvious, ideal example once explained, and yet, every person where I worked who was unfamiliar with Erlang (and even some of those who had coded a little in it, but hadn't come to grasp the paradigm as well), who was explained what we were trying to do, described it as "easy, we just need to use a priority queue and pull from it!"

This assumes a pedantic qualification of "big data" handling -- does storage count, as opposed to the actual processing of the data? But I think that's an important distinction in this context, as it strikes the heart of the dichotomy between concurrency and parallelism.

(Moreover, I've never really been sure how much Riak is being used for actual 'big data', as opposed to being a master-less, highly available repository for 'regular data' (distinction between 'big' and 'regular' deliberately left vague). More so since Riak is key-value as opposed to columnar, but I suppose that depends on your workload. But that's all besides the point here.)