Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BEAM languages, Hindley–Milner type systems, and new technologies

medium.com

Post reply on HN
Print view View original

41–50 of 87 posts

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#41

Post by pron »

corysama wrote: post #17
pron wrote: post #14

Earlier quoted context omitted.

> Erlang makes it much easier to do well and much harder to screw up. That's a feature of the language, not the VM (compare with Clojure, that does a similar thing on the JVM). You could still do all that on a higher-quality VM (simply because the effort put into it is orders-of-magnitude more than into BEAN; not because OpenJDK's people are smarter or anything).

If I could get the JVM's JIT & serial GC performance combined with the BEAM's trivial-cost threads & thread-segregated GC, it would be sweet indeed.

> serial GC performance

HotSpot hardly ever uses a serial GC anymore. It's now parallel or parallel and concurrent.

> thread-segregated GC

You don't really want that if a shared-heap GC can buy you better performance because it's more mature and saves you all the copying.

> BEAM's trivial-cost threads

You can have that on the JVM.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#42

Post by pron »

seiji wrote: post #21
pron wrote: post #14

Earlier quoted context omitted.

> Erlang makes it much easier to do well and much harder to screw up. That's a feature of the language, not the VM (compare with Clojure, that does a similar thing on the JVM). You could still do all that on a higher-quality VM (simply because the effort put into it is orders-of-magnitude more than into BEAN; not because OpenJDK's people are smarter or anything).

That's a feature of the language, not the VM Lies. One way to look at the Erlang SMP VM is basically as a load balancer. Erlang automatically migrates processes between cores for maximum concurrent efficiency. It's basically coordinating N "tiny" Erlang VMs across all your cores and knows how to, ideally, optimally place your workload. You can even constrain the behavior to a per-core and per-scheduler level with VM…

> Erlang automatically migrates processes between cores for maximum concurrent efficiency. It's basically coordinating N "tiny" Erlang VMs across all your cores and knows how to, ideally, optimally place your workload. You can even constrain the behavior to a per-core and per-scheduler level with VM options—not language options. See the +S and +SP and +SDcpu and +SDPcpu and +SDio and +sct options at http://www.erlang.org/doc/man/erl.html

That's amazing, except that that's the work of Erlang's wrok-stealing scheduler, and as it happens, the JDK currently has the best work-stealing scheduler around.

> Our bottlenecks these days are programmer time and programmer thought correctness. Generating more work for programmers by making them write lower level code isn't the way forward even if the more work is slightly faster.

Why more work? I am for Erlang. Keep using Erlang. Just run it on the JVM. It's the same work with better results.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#43

Post by pron »

nickpsecurity wrote: post #31
pron wrote: post #2

BEAM is a very nice VM (albeit rather slow compared to HotSpot or V8), but I don't understand why every mention of BEAM has to spread misconceptions about the JVM: > In many systems, Java included, the Garbage Collector (GC) must examine the entire heap in order to collect all the garbage. There are optimizations to this, like using Generations in a Generational GC, but those optimizations are still just optimization…

What you're saying is a VM with tons of R&D, tons of corporate investment, and a focus on speed was faster than a new one that was Ericsson's side project focused on stuff other than speed? Little surprise. Meanwhile, BEAM and its language have been doing exactly what they're designed for with enough success that it's mainstreamed naturally. Which Java didn't. Truth be told, most of the crowd using BEAM doesn't care…

> avoiding the bullseye currently on Java.

That bullseye exists only in the minds of some HNers. Here is a very (very!) partial list of companies running primarily or largely on the JVM: Google, Twitter, Netflix, LinkedIn, Box, IBM, SAP, Amazon, eBay.

> They just want easy scaling, distribution, and fault-tolerance.

... So they write chunks of their code in C. That would be completely unnecessary if they'd just run Erlang on the JVM.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#44

Post by pron »

dozzie wrote: post #34
pron wrote: post #13

Earlier quoted context omitted.

Sure, but you can have those millions of processes on the JVM, too and scale better because of better access to shared data than ETS.

> [...] but you can have those millions of processes on the JVM, too [...] In theory, yes. In practice, it's very difficult. JVM thread corresponds to a thread of host OS. Spawning and keeping those is very slow and expensive compared to Erlang's processes. You could have a pool of pre-spawned workers, but suddenly you can't spawn a worker for each connection and hope it will all work; you need to manage the pool. Yo…

> you need to manage the pool. You also could try to implement green threads in JVM, as they are in Erlang, but you would need an entirely new compiler to insert yield points at appropriate places.

I am talking about a new compiler. An Erlang compiler. The JVM is a virtual machine. Erlang is a language. We've already proven you can run Erlang on the JVM quite well, and that was before many pertinent improvements to the JVM and its ecosystem.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#45

Post by nickpsecurity »

pron wrote: post #43
nickpsecurity wrote: post #31

Earlier quoted context omitted.

What you're saying is a VM with tons of R&D, tons of corporate investment, and a focus on speed was faster than a new one that was Ericsson's side project focused on stuff other than speed? Little surprise. Meanwhile, BEAM and its language have been doing exactly what they're designed for with enough success that it's mainstreamed naturally. Which Java didn't. Truth be told, most of the crowd using BEAM doesn't care…

> avoiding the bullseye currently on Java. That bullseye exists only in the minds of some HNers. Here is a very (very!) partial list of companies running primarily or largely on the JVM: Google, Twitter, Netflix, LinkedIn, Box, IBM, SAP, Amazon, eBay. > They just want easy scaling, distribution, and fault-tolerance. ... So they write chunks of their code in C. That would be completely unnecessary if they'd just run E…

Oh no, it exists in the form of CVE's and actual compromises. Java compromises were coming into my news feed at a higher rate than Windows despite all its native code. Got bad enough that Krebs on Security simply recommended taking Java off one's machine unless they absolutely need it. Nice that you named many top tech firms with smart, well-funded NOC's and security teams as the counter-example. Harder for individuals and smaller firms than simply installing what works and hardly anyone is attacking. ;)

On second point, Erlang on the JVM currently gives same scaling, real-time properties, easy distributed apps, and availability as BEAM? And without one of the commercial VM's you seem to be assuming (but not stating) for eg real-time? If so, you might have a strong argument on that end. You just need to demonstrate each with examples of real-world apps running in Erlang on each. More people would accept your claim if you demonstrated it.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#46

Post by eggy »

jdimov9 wrote: post #25

Here's a tool that you can play with to see how well Elixir scales with an embarrassingly parallel task (matrix multiplication) when throwing more CPU cores at it: https://github.com/a115/exmatrix

Yes, Elixir does well here, but I still prefer Lisp syntax. I would like to see a comparison of LFE and Joxa. Joxa seems more like Clojure. This presentation is a good one, but I'd like to see a nut and bolts comparison with side-by-side code:

http://www.slideshare.net/BrianTroutwine1/erlang-lfe-elixir-...

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#47

Post by anonyfox »

pron wrote: post #40
cdumler wrote: post #18

Earlier quoted context omitted.

No, Erlang does not mean that anything you write will scale. Your solution has to be broken into parallelizable pieces. But, the scalability of your solution is only as good as the mechanisms within the language and runtime to efficiently allow developers to create a scalable system. BEAM implements several nice features: Data is immutable, so we don't have to worry about keeping data coherent between.. anything. Whe…

> Data is immutable... But you don't need that at the VM level. Clojure does that on the JVM. Having that at the VM level makes a simple GC work reasonably well, but HotSpot has world-class GCs that perform better, even without the assumption of immutability. > Everything is defined in modules. Again, that's a language-level feature. > Processes are an abstraction You can get that on the JVM, too. > Finally, everythi…

Hey man, some people (like me) just don't like to work on the JVM, despite it's advantages and superior features like the GC. Just accept it.

Having worked with java, scala, jruby and closure, something always is clumsy, be it interfacing with java cruft, slow startup times of the vm, maven & co... While there are solutions to fix the solutions, its just annoying for me. I get what you say, but nevertheless are JVM (and .NET) based things nothing i'd use (except i'm forced to do so).

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#48

Post by pron »

vezzy-fnord wrote: post #15
pron wrote: post #13

Earlier quoted context omitted.

Sure, but you can have those millions of processes on the JVM, too and scale better because of better access to shared data than ETS.

Erlang generally encourages shared-nothing architectures. Of course, in some cases you want regions of shared memory or some other concurrent global resource, hence ETS. I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates. Scalability and large actor counts aren't the definitive features of Erlang, though. It's supervision trees, the distributi…

> There is absolutely no reason to switch to the JVM when the EVM is a beast of its own.

I think there is, if you want to concentrate your limited resources on the language and its phenomenal libraries while letting an enormous team working on the world’s second-largest open-source project take care of the VM for you, while at the same time giving you better performance and a wider reach. There are many more organizations that would adopt Erlang if it were on the JVM.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#49

Post by pron »

nickpsecurity wrote: post #32
pron wrote: post #10

Earlier quoted context omitted.

> Actor model does not require concurrent data structure to be effective. Actually that's the whole point of it... That's complicated. Actors don't require concurrent data structures for the bits that don't require concurrent data structures but they do for the bits that do :) That's why Erlang has ETS. That's why you still need a database.

No it's not. The proper way to design systems is breaking them into pieces, implementing each one the best way for that piece, and connecting them together. So, if one can't do DB's with Actors, do it with another tool or model (eg Eiffel/Java w/ SCOOP). Use Erlang/BEAM for what it's good at. There's also tools such as ZeroMQ that make the integration fast & easier. You get the best of both worlds.

All the more reason to use Erlang/JVM alongside other JVM languages. Polyglotism is one of the JVM's greatest strengths, and interlanguage interoperability on the JVM is the best you can get.

And Erlang/JVM can be even better than Erlang/BEAM at everything Erlang/BEAM is good at. There is nothing in BEAM that makes it more appropriate for running Erlang than HotSpot (that may have been true in the past, but that's no longer the case). BEAM's Erlang specialty simply means its development required relatively little effort to run Erlang reasonably well. It doesn't mean the JVM can't run Erlang better, and at this point in time we have every reason to believe it can run Erlang much better than BEAM.

Re: BEAM languages, Hindley–Milner type systems, and new technologies

#50

Post by pron »

nickpsecurity wrote: post #37
chrisseaton wrote: post #19

Earlier quoted context omitted.

I can keep quoting from the same book: > Here’s the good news: your Erlang program might run n times faster on an n-core processor—without any changes to the program. Sounds hopeful - and they've qualified it with might which is good. > But you have to follow a simple set of rules. If you want your application to run faster on a multicore CPU, you’ll have to make sure that it has lots of processes, that the processes…

That's not true for any language. It depends heavily on how that language is interpreted or compiled to machine code. A number of languages support threads but have eg GIL's. We have to judge each on a case-by-case basis. Both the Erlang language and BEAM designed specifically to support this. Here's some details for you in a JVM comparison: http://ds.cs.ut.ee/courses/course-files/To303nis%20Pool%20.p...

Unfortunately, that comparison lacks a lot of pertinent information on the JVM like new GCs, new schedulers, new, better JITs and various lightweight-thread implementations (it does mention Quasar, but doesn't understand that it works just like BEAM. BEAM also instruments your code, and in BEAM you can also accidentally block an entire kernel thread by calling a library not written in Erlang: just as you would if you were running Erlang on the JVM.

Like I said in another comment, BEAM's Erlang specificity does not mean that it's the best VM for Erlang; all data points to HotSpot (today) being a much better Erlang VM. It just means that a reasonable VM for Erlang could be developed with relatively little effort.

Post reply on HN