BEAM languages, Hindley–Milner type systems, and new technologies

Nice article, and for an Elixir fan, provides a nice little snippet on something I've been having issues with but hadn't really put my finger on until I saw it:

"[...] and I really dislike that Elixir tries to hide immutability. That does make it slightly easier for beginners, but it’s a leaky abstraction. The immutability eventually bleeds through and then you have to think about it."

I don't think it necessarily tries to hide it (at all), but it does have some instances where something feels like a mutable structure. Those can be, at least for me, a bit confusing to reason about if you're expecting things both be and look immutable.

I suppose now that I know exactly what's weird, I should just go dig through the code and figure it out. Problem solved?

... One other thing, because I see this in the comments already, is that BEAM isn't the tool for every job-- but for some jobs, it is the only tool to do it well. Is the JVM faster at general tasks? Hell yes, but that's not the point, it's not even why BEAM is around.

It's about:

* Small concurrent workloads. Really long running CPU intensive tasks aren't going to be good.

* Low latency. Not just low, but with a very very small standard deviation. Your application's performance will be consistent.

* Fault tolerant.

The list goes on, and here's a nice summary of it (both bad and good):

http://blog.troutwine.us/2013/07/10/choose_erlang.html

There are times when I choose the JVM, there are times when I choose BEAM or MRI. I just try choose the right tool for the job, but some tools, make some jobs, very difficult.

cough ruby cough concurrency cough

Edit: One thing for people not familiar with BEAM, a "process" is not a Unix process, from the Elixir documentation:

"Processes in Elixir are extremely lightweight in terms of memory and CPU (unlike threads in many other programming languages). Because of this, it is not uncommon to have tens or even hundreds of thousands of processes running simultaneously."

> Erlang makes it much easier to do well and much harder to screw up.

That's a feature of the language, not the VM (compare with Clojure, that does a similar thing on the JVM). You could still do all that on a higher-quality VM (simply because the effort put into it is orders-of-magnitude more than into BEAN; not because OpenJDK's people are smarter or anything).

> brilliantly to run in parallel.

I just want to clarify it's for concurrent not parallel.

Erlang doesn't promise parallel. You can get parallel from concurrent but not the other way around and once again Erlang only enable concurrency and you may get parallel beacuse of concurrency.

No, Erlang does not mean that anything you write will scale. Your solution has to be broken into parallelizable pieces. But, the scalability of your solution is only as good as the mechanisms within the language and runtime to efficiently allow developers to create a scalable system. BEAM implements several nice features:

Data is immutable, so we don't have to worry about keeping data coherent between.. anything. Whether it is two processes or two nodes. New data is can be constructed with reference to old data without fear that the old data will be modified. So, "mutation" is really just new data with a reference to the old unchanged data. This greatly lowers the churn in creating new data. It also means everything can just pass (process to process or node to node) what it has without feature it will be out of date.

Everything is defined in modules. Modules define what we would think of in OOP as namespaces, structures, classes/types, and class functions. Importantly, they only define functionality. Modules do not have state. Therefore, functions accept some set of inputs, create new data from the inputs (no mutation), and return some output. This makes it very easy reason about what the code is doing if you keep the modules well defined and reasonably sized. This code can be shared around easily, too. It's got no state and is immutable.

Processes are an abstraction. You can think of them as thread, but they're really just a stack and and a little book keeping. A BEAM VM will normally of real threads equal to the number of CPUs in the machine. Each real thread will then exclusively pick a process, load the book keeping, point itself to the stack, and execute bytecode for a period of time. When done, it will mark the changes in the book keeping, and move to the next process. This is very lightweight, so literally millions can run on a single computer. Because they are self contained, they're easy to clean up. Processes also expose standard set of interfaces for communication, a pub/sub system. Again, immutable messages are sent back and forth. So, it doesn't matter if it's the same node or not.

Finally, everything is abstracted to the notion of nodes with in a cluster. By default, anything you executes on the local node, but you can specify otherwise. I can execute a module call on another machine or spawn a new process on another machine. It just means a little more information in the call, but it's the same exact concept programmatically. Also, it's possible group processes into named services. You can call a named service and it will know what processes to contact. It's a very low barrier to entry to parallelize your code if you just write it that way.

When you start thinking in terms of how structure you code for BEAM, you inherently get easy access to scalability.

When a NIF causes a crash it does take down the whole VM. I think the way to isolate NIFs is have them on their own nodes.