Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BEAM languages, Hindley–Milner type systems, and new technologies

medium.com

Post reply on HN
Print view View original

21–30 of 87 posts

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#21

Post by seiji »

pron wrote: post #14
corysama wrote: post #11

Earlier quoted context omitted.

Everything you are saying is technically correct. The issue is that Erlang is trying to solve a different problem than you are describing. It sounds like you are hoping to perform some large but single task and are disappointed that Erlang can't defeat the Amdahl limitations inherent in your task. That's not Erlang's goal. Erlang's goal is to take problems that are embarrassingly parallel in theory and make them emba…

> Erlang makes it much easier to do well and much harder to screw up. That's a feature of the language, not the VM (compare with Clojure, that does a similar thing on the JVM). You could still do all that on a higher-quality VM (simply because the effort put into it is orders-of-magnitude more than into BEAN; not because OpenJDK's people are smarter or anything).

That's a feature of the language, not the VM

Lies.

One way to look at the Erlang SMP VM is basically as a load balancer. Erlang automatically migrates processes between cores for maximum concurrent efficiency. It's basically coordinating N "tiny" Erlang VMs across all your cores and knows how to, ideally, optimally place your workload. You can even constrain the behavior to a per-core and per-scheduler level with VM options—not language options. See the +S and +SP and +SDcpu and +SDPcpu and +SDio and +sct options at http://www.erlang.org/doc/man/erl.html

There are tradeoffs between being the best language for a task and being the fastest language for a task. The more work you can move into the VM, the less you have to do as an application programmer, but potentially the slower your program may go since the VM has to discover or introspect your actions instead being told explicitly.

Our bottlenecks these days are programmer time and programmer thought correctness. Generating more work for programmers by making them write lower level code isn't the way forward even if the more work is slightly faster.

All that being said, everything has a price. Obviously never do numeric computing work in regular Python. Grab you some numpy or GPU frameworks. In the same vein, never do massively concurrent programming without Erlang or without a highly optimized event loop (but with an event loop you're limited back to one core, and on modern 48+ core systems, that's kinda pathetic).

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#22

Post by kasey_junk »

vezzy-fnord wrote: post #15
pron wrote: post #13

Earlier quoted context omitted.

Sure, but you can have those millions of processes on the JVM, too and scale better because of better access to shared data than ETS.

Erlang generally encourages shared-nothing architectures. Of course, in some cases you want regions of shared memory or some other concurrent global resource, hence ETS. I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates. Scalability and large actor counts aren't the definitive features of Erlang, though. It's supervision trees, the distributi…

> Erlang generally encourages shared-nothing architectures.

This is the language feature that `pron` keeps mentioning. Nothing about the VM is especially better for this than the JVM for instance.

> I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates.

There isn't anything wrong with it (as a complete neophyte to ETS and the EVM generally), the question is how much better it could be if it was on one of the several first rate JVMs that get so much more resources poured into them. Sharing data concurrently is precisely what the JVM is good at (especially at very large data set size). So in the cases where you need to use something like ETS, there is a lot of potential for improvement on a JVM vs EVM.

> But it's already there in a cohesive whole. There is absolutely no reason to switch to the JVM when the EVM is a beast of its own.

I don't want to speak for `pron` but I suspect what he is getting at is, the combination of the Erlang full story on the JVM would be a phenomenal bit of tech and it would be much easier (and more likely) for the Erlang bits to get ported to the JVM than it would be to bring the EVM up to the standard of any of the best JVMs.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#23

Post by jsprogrammer »

anonyfox wrote: post #4
pron wrote: post #2

BEAM is a very nice VM (albeit rather slow compared to HotSpot or V8), but I don't understand why every mention of BEAM has to spread misconceptions about the JVM: > In many systems, Java included, the Garbage Collector (GC) must examine the entire heap in order to collect all the garbage. There are optimizations to this, like using Generations in a Generational GC, but those optimizations are still just optimization…

The point of erlang isn't beating micro benchmarks. Everyone and his dog knows that other techs have better performance. The ease of scaling across machines, fault tolerance and low latency variation are more typical selling points. Besides that, god prevent erlang to become just-another-JVM-language, I embrace competition.

Do you know of a chart that compares all of these dimensions across languages/VMs?

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#24

Post by seiji »

sjtgraham wrote: post #20
NhanH wrote: post #8

Earlier quoted context omitted.

> True, that is a good selling point -- in theory. Indeed BEAM's process isolation is better than the JVM's on paper. In practice, so many Erlang systems have so much C in them (because Erlang isn't fast enough for the data plane), that they can still bring down the entire VM (not as if there aren't other ways of doing that even without native code), or they interfere with one another in other ways because of BEAM's…

When a NIF causes a crash it does take down the whole VM. I think the way to isolate NIFs is have them on their own nodes.

Kinda hilarious since we've had completely isolated ports for a loooong time. For non-data-intensive tasks, don't underestimate how fast speaking erl_interface over a unix pipe can be. Plus, free isolation, free fault tolerance and free supervision restarting without any networking required.

http://erlang.org/doc/reference_manual/ports.html (15.1)

Here's one of my old examples (no guarantees to its current effectiveness or correctness): https://github.com/mattsta/libgeoip-erlang/blob/master/c_src... — then the whole thing is opened and run from Erlang like https://github.com/mattsta/libgeoip-erlang/blob/70b58ef5ef8a...

It's just cleaner to stay outside direct VM linkage as much as possible.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#27

Post by pests »

chrisseaton wrote: post #19
digitalzombie wrote: post #16

Earlier quoted context omitted.

> brilliantly to run in parallel. I just want to clarify it's for concurrent not parallel. Erlang doesn't promise parallel. You can get parallel from concurrent but not the other way around and once again Erlang only enable concurrency and you may get parallel beacuse of concurrency.

I can keep quoting from the same book: > Here’s the good news: your Erlang program might run n times faster on an n-core processor—without any changes to the program. Sounds hopeful - and they've qualified it with might which is good. > But you have to follow a simple set of rules. If you want your application to run faster on a multicore CPU, you’ll have to make sure that it has lots of processes, that the processes…

Per your last comment - that is what he was saying. Vector instruction sets allow parallel, which does not imply concurrency.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#28

Post by chrisseaton »

pests wrote: post #27
chrisseaton wrote: post #19

Earlier quoted context omitted.

I can keep quoting from the same book: > Here’s the good news: your Erlang program might run n times faster on an n-core processor—without any changes to the program. Sounds hopeful - and they've qualified it with might which is good. > But you have to follow a simple set of rules. If you want your application to run faster on a multicore CPU, you’ll have to make sure that it has lots of processes, that the processes…

Per your last comment - that is what he was saying. Vector instruction sets allow parallel, which does not imply concurrency.

Ah yes you're right sorry. I was thinking without reading carefully that he was saying you can't have parallelism without concurrency.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#29

Post by Ixiaus »

kasey_junk wrote: post #22
vezzy-fnord wrote: post #15

Earlier quoted context omitted.

Erlang generally encourages shared-nothing architectures. Of course, in some cases you want regions of shared memory or some other concurrent global resource, hence ETS. I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates. Scalability and large actor counts aren't the definitive features of Erlang, though. It's supervision trees, the distributi…

> Erlang generally encourages shared-nothing architectures. This is the language feature that `pron` keeps mentioning. Nothing about the VM is especially better for this than the JVM for instance. > I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates. There isn't anything wrong with it (as a complete neophyte to ETS and the EVM generally), the…

It would be a phenomenal bit of tech anywhere. We had to go with Erlang for one piece of our product at Plum precisely because there is no analog story in Haskell to the Erlang full story. I would have loved to be able to build that specific piece in Haskell but it made little sense when considering what Erlang/OTP provides.

It's nice to say that the JVM has more resources and is better at XYZ while the BEAM VM is only better at ABC, therefore the Erlang full story should be on the JVM to reap the benefits of both; however, I think that would be unhealthy for Erlang. Different VMs present specialized focuses and I think the areas that BEAM is lacking in can be tackled and brought up to parity instead homogenizing the VM-field and adding to the kitchen-sink that the JVM already is.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#30

Post by nickpsecurity »

abrgr wrote: post #26

"It’s not going to be too much longer before we declaratively describe out systems as well as our code. I am looking forward to that." Amen! Been doing that to the extent possible for a while and it is terrific!

It's been done before to varying degrees. The original in automatic programming was one that took input from case tools and autogenerated a lot of COBOL. Sun's DASL language from ACE project was a domain-specific language for specifying a type of web application. Around 9-10kloc of it autogenerated 100+kloc XML, client code, server code, etc. Lots of them in the 4GL category for database manipulation with WINDEV/WEBDEV more general-purpose yet still requiring coding in BASIC-like language.

So, it's not far-fetched. It will likely be a series of DSL's like the above or iMatix's model-driven development approach. These would specify it at a high level with precise requirements and constraints. Then, planning software with heuristics would produce the code. Similar systems for integration. Several people's worth of work or 10-20 tools becomes one person with one set of tools. Doubt we'll replace the person or need for some programming tools.

Post reply on HN