Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

ds.cs.ut.ee

Post reply on HN
Print view View original

51–60 of 60 posts

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#51

Post by polack »

abalone wrote: post #44
polack wrote: post #37

Earlier quoted context omitted.

It also shows that Erlang is 15X faster than Java (depending on what framework you compare) and that the Java Netty/JSON test has 8X more code than the Erlang Cowboy/JSON counterpart ;)

>It also shows that Erlang is 15X faster than Java That's nonsense. jlouis challenged us to "Take the fastest webserver written for Java and for Erlang", not the slowest ones. And that's exactly what the TechEmpower benchmarks do and they give Java a ~500% advantage. As for codesize, if you don't like netty then the very next ranked result is servlet3-cass which delivers nearly the same performance -- again, approx 5…

There was a smiley in the end you know... My point was that these "tests" really shows nothing. It's a very specific scenario and if you read the comment from jlouis you would know that his scenario was very different and would yield a different result.

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#52

Post by azth »

pron wrote: post #50
digitalzombie wrote: post #10

Earlier quoted context omitted.

> I believe that the JVM is a strict superset of any Erlang VM. I don't get how JVM is a strict superset of Erlang VM when it doesn't have pre-emption. If BEAM set have preemptive scheduler as an element and JVM, IIRC, does not have such feature, then JVM is not a super set let alone a strict super set.

Quasar does preemptive scheduling, and even used to have time-slice based preemption. The latter was taken out as it proved to provide no benefit whatsoever, because processes that benefit from time-slice preemption are better off using the kernel's scheduler anyway, and the JVM gives them that option. Erlang has to do it because it only exposes a single scheduler. That's is still a strict subset of the JVM's capabil…

> and the JVM gives them that option

By using java.lang.Thread?

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#53

Post by phamilton »

pron wrote: post #46
dschiptsov wrote: post #41

Earlier quoted context omitted.

You cannot give "similar behavior" of lock-free, share-nothing data-structures with locking-concurrent ones.

The JDK has some of the best implementations of lock-free data structures anywhere, and they are actually all implemented in Java (the JVM exposes direct access to memory fences and CAS). You really never have to drop down to C unless you want to interface with some OS-specific code. The JVM is simply much more general-purpose. Just like your machine and OS themselves can support Erlang, so can the JVM.

So is this a sort of Greenspun's tenth rule situation? Any sufficiently advanced distributed systems contains a buggy implementation of Erlang?

More seriously, is that what you are suggesting? By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc you end up with all the benefits of Erlang while maintaining the multi-paradigm platform of the JVM? Sounds amazing. Give it to me with out of the box sane configuration and I'm sold.

It sounds like it's not quite the case, especially around distributed systems. But perhaps that's just the intended design because clustering machines is sort of out of style in the age of RESTful microservices.

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#54

Post by pron »

phamilton wrote: post #53
pron wrote: post #46

Earlier quoted context omitted.

The JDK has some of the best implementations of lock-free data structures anywhere, and they are actually all implemented in Java (the JVM exposes direct access to memory fences and CAS). You really never have to drop down to C unless you want to interface with some OS-specific code. The JVM is simply much more general-purpose. Just like your machine and OS themselves can support Erlang, so can the JVM.

So is this a sort of Greenspun's tenth rule situation? Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? More seriously, is that what you are suggesting? By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc you end up with all the benefits of Erlang while maintaining the multi-paradigm platform of the JVM? Sounds amazing. Give it to me with out of the box sane…

> Any sufficiently advanced distributed systems contains a buggy implementation of Erlang?

Why won't you say that about machine code? The JVM simply operates at a lower level than Erlang, hence it can be used to implement Erlang, and, it turns out that doing so might gives better results than current Erlang VMs implemented in C, because there are man-decades (or centuries) of shared functionality that's already in the JVM.

> By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc

You don't need to rewrite or heavily configure the GC (besides, which one? There are plenty of JVM GCs) -- just to use it as Erlang does, or one that is pauseless no matter how you use it (two implementations, one commercial and one open-source under development at Red Hat). Also, you don't need to add a scheduler. One of the world's best implementations of a work-stealing scheduler, also with about a decade of effort behind it by Doug Lea, is packaged with the JDK. See here[1] to get a feel for the effort behind it.

> It sounds like it's not quite the case, especially around distributed systems.

Erlang doesn't add anything not already found in high-quality, battle-tested libraries like JGroups[2] when it comes to clustering (actually, JGroups is more powerful than Erlang's clustering). The JVM ecosystem is so huge that it's very hard to come up with something (other than syntax) that doesn't already have a solid implementation out there.

I'd really like to retrofit Erjang to use Quasar rather than Kilim (Quasar is just better maintained, more battle-tested and probably faster).

I really believe that running Erlang on the JVM is a win for everyone. The Erlang community simply doesn't have a tenth of the resources that are put into OpenJDK[3]. Why not enjoy the benefits?

[1]: https://www.youtube.com/watch?v=sq0MX3fHkro

[2]: http://www.jgroups.org/

[3]: like a groundbreaking optimizing compiler that will start being used in Java 9, and can be especially appropriate for compiling Erlang: https://wiki.openjdk.java.net/display/Graal/Publications%20a...

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#55

Post by pron »

dxhdr wrote: post #20
pjmlp wrote: post #18

Earlier quoted context omitted.

Try that with Zing. The beauty of the JVM, is that by being a specification, there are lots of implementations to choose from. Yet people seem to think OpenJDK is the only one.

Sure I'll just fork over $8,000 and get started... The beauty of BEAM is that I don't have to know or worry about projects like Zing, BEAM "just works." (Another mark against Zing just because you brought it up -- where's the source code?)

It's not that you don't have to know or worry about groundbreaking projects like Zing, but that they don't exist. The resources put into advancing the Erlang ecosystem are a tiny fraction of those put into the JVM ecosystem, and it shows.

You say BEAM "just works", but it "just works" if you can put up with its non-stellar performance or happen to play directly to its strengths. Any technology just works if you happen to need exactly what it provides. Java and the JVM, however, like the Linux kernel, are a huge project with vast resources, wide reach and very wide applicability from small embedded devices to mainframes. So, naturally, there are a lot of things to choose from, but you are guaranteed to find one that suits your need -- no matter what it is.

Too many Erlang projects start out thinking Erlang is good enough, and then find that they need to write more and more of their code in C. This just cannot happen with the JVM. Not that it's appropriate for everything (it's bad for command-line utilities and Java SE is inappropriate for constrained environments), but it's much harder to miss. You don't need to know or worry about Zing unless you need what it provides (worst-case latency of under 50us); similarly for other tens of thousands of JVM libraries and tools. All you need to know is that you can find a high-quality implementation of whatever you might need down the line.

As for Zing's source code: Zing is proprietary, but as the JVM ecosystem is so big, there's a similar open-source attempt by Red Hat and intended to go into OpenJDK called Shenandoah[1]. Its code is here: http://icedtea.classpath.org/hg/jdk9-shenandoah/hotspot/

[1]: http://openjdk.java.net/jeps/189

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#56

Post by pron »

azth wrote: post #52
pron wrote: post #50

Earlier quoted context omitted.

Quasar does preemptive scheduling, and even used to have time-slice based preemption. The latter was taken out as it proved to provide no benefit whatsoever, because processes that benefit from time-slice preemption are better off using the kernel's scheduler anyway, and the JVM gives them that option. Erlang has to do it because it only exposes a single scheduler. That's is still a strict subset of the JVM's capabil…

> and the JVM gives them that option By using java.lang.Thread?

Yep. Both Thread and Fiber are abstracted into a Strand. So Strand.currentStrand returns the current fiber/thread, Strand.sleep sleeps etc.. Locks and channels work with strands, too, so that both fibers and threads can synchronize on the same data, and even actors are assigned to a strand, so that an actor can run in either a fiber or a thread.

Then, you pick the strand implementation (lightweight or heavyweight) -- heavyweight for computation-heavy code like video encoding, lightweight for anything that blocks often -- and if you pick a fiber you can further select a particular scheduler, with the default being work-stealing (like in Erlang or Go), but you can plug your own (and control such stuff as CPU pinning for certain schedulers).

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#57

Post by abalone »

polack wrote: post #51
abalone wrote: post #44

Earlier quoted context omitted.

>It also shows that Erlang is 15X faster than Java That's nonsense. jlouis challenged us to "Take the fastest webserver written for Java and for Erlang", not the slowest ones. And that's exactly what the TechEmpower benchmarks do and they give Java a ~500% advantage. As for codesize, if you don't like netty then the very next ranked result is servlet3-cass which delivers nearly the same performance -- again, approx 5…

There was a smiley in the end you know... My point was that these "tests" really shows nothing. It's a very specific scenario and if you read the comment from jlouis you would know that his scenario was very different and would yield a different result.

You're trying to dismiss the benchmark because you don't like the result. But actually the tests do show max latency with high concurrency which is the question at hand, and AFAICS the only factor not covered is a minutes-long test that might surface any GC impacts. So they are an excellent base for comparing performance.

Again if you or anyone can prove a significant GC impact serving up text or Json definitely hit up the techempower folks and make a case for extending the test length. It might help improve Erlang's standing which right now utterly sucks in comparison to Java performance (probably because of the maturity of the JVM).

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#58

Post by phamilton »

pron wrote: post #54
phamilton wrote: post #53

Earlier quoted context omitted.

So is this a sort of Greenspun's tenth rule situation? Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? More seriously, is that what you are suggesting? By rewriting (or heavily reconfiguring) the GC, adding a scheduler, etc you end up with all the benefits of Erlang while maintaining the multi-paradigm platform of the JVM? Sounds amazing. Give it to me with out of the box sane…

> Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? Why won't you say that about machine code? The JVM simply operates at a lower level than Erlang, hence it can be used to implement Erlang, and, it turns out that doing so might gives better results than current Erlang VMs implemented in C, because there are man-decades (or centuries) of shared functionality that's already in th…

Greenspun's tenth rule is tongue-in-cheek and usually followed by something like "...including Erlang itself".

I get it. I see the huge benefits in a ground-up design for Erlang in the JVM. (Said design should definitely use existing GC, scheduler, etc.).

However, while what you are describing is technically the JVM, it's sort of a Ship of Theseus (with the GC replaced, with static binary manipulation by Quasar, etc.). This isn't a bad thing, but it's not what I'd normally associate with the JVM and its ecosystem.

I've got a bit of a "chip on my shoulder" regarding what I see as a pretty terrible experience with Akka. I feel strongly that preemption makes a huge difference in usability. With Akka, there are too many caveats relating to things you can and cannot do. Rather than simplifying my life, Akka gives me a whole bunch of additional ways I can break my application. While it could be said that "being a professional engineer" means solidly understanding my tools, I don't feel that is a scalable way to build development teams.

I'd love an out-of-the-box "erlang mode" switch (or some sort of launcher) that would give me these safe behaviors. Something along the lines of Percona for MySQL.

Reach out to me directly with email (in profile). This has been a fascinating discussion and I'm very interested in learning more.

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#59

Post by pron »

phamilton wrote: post #58
pron wrote: post #54

Earlier quoted context omitted.

> Any sufficiently advanced distributed systems contains a buggy implementation of Erlang? Why won't you say that about machine code? The JVM simply operates at a lower level than Erlang, hence it can be used to implement Erlang, and, it turns out that doing so might gives better results than current Erlang VMs implemented in C, because there are man-decades (or centuries) of shared functionality that's already in th…

Greenspun's tenth rule is tongue-in-cheek and usually followed by something like "...including Erlang itself". I get it. I see the huge benefits in a ground-up design for Erlang in the JVM. (Said design should definitely use existing GC, scheduler, etc.). However, while what you are describing is technically the JVM, it's sort of a Ship of Theseus (with the GC replaced, with static binary manipulation by Quasar, etc.…

But you don't need to replace the GC. HotSpot alone comes with a choice of 3 (or 4) GCs, and a fifth will join them. As of Java 9, HotSpot will support pluggable JITs (written in Java!). Quasar's bytecode manipulation is common practice among a large number of JVM tools, including one of the most popular Maven plugins, many of the most popular JVM profilers, and Spring. It isn't some exotic technique.

And don't forget -- the JVM is a spec with many implementations (including some for hard-realtime systems), although I'm talking of OpenJDK's HotSpot here.

I totally agree about preemption (and Akka, obviously). Blocking a thread (userspace or kernel) is the essential abstraction of imperative programming (or, more accurately, suspending a continuation) of that is the essence of imperative programming (even functional imperative programming, like Erlang and Clojure).

Anyway, I'll get in touch :)

Re: Comparison of Erlang Runtime System and Java Virtual Machine [pdf]

#60

Post by dschiptsov »

pron wrote: post #46
dschiptsov wrote: post #41

Earlier quoted context omitted.

You cannot give "similar behavior" of lock-free, share-nothing data-structures with locking-concurrent ones.

The JDK has some of the best implementations of lock-free data structures anywhere, and they are actually all implemented in Java (the JVM exposes direct access to memory fences and CAS). You really never have to drop down to C unless you want to interface with some OS-specific code. The JVM is simply much more general-purpose. Just like your machine and OS themselves can support Erlang, so can the JVM.

In Armstrong Thesis there are explicit assertion about why thread-based JVM is a flawed design.

My point was that there is a big difference between a VM for a functional and imperative language (immutability makes it and GC a lot simpler). JVM's selling points, that it is "general" and is "already here" are strong, but in my opinion, its praise is more religion-like repetition of dogmas and marketing memes.

I would like to assert that Erlang designers have made better design decisions (to avoid threading and mutation whatsoever) so Erlang's VM is much simpler, more stable (JVM crashes often, wastes memory - memory used vs. memory served ratio, and GC behavior tend to become unpredictable for long-running process) and good-enough.

Post reply on HN