Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Great Ruby Shootout (July 2010)

programmingzen.com

Post reply on HN
Print view View original

21–30 of 45 posts

Re: The Great Ruby Shootout (July 2010)

#21

Post by acangiano »

headius wrote: post #19

I can comment on the JRuby results. For the base perf numbers, I'm not surprised. We've known that we're roughly on par with 1.9.2 for some time, and many of the benchmarks in question have started to reach a point of irreducible complexity (e.g. you can only slice and dice strings so fast). It's good to see JRuby remains at or near the front of the pack as far as performance, especially considering we've made no maj…

> The Linux versus Windows numbers are a bit surprising to me. If I were to make a guess, I'd guess that the JVMs used were not identical (perhaps like Isaac Gouy mentions, one platform was 64-bit and the other wasn't) or some other detail altered the performance characteristics of the test.

The same identical version was used: Java HotSpot(TM) 64-Bit Server VM 1.6.0_20.

> But the performance drop does seem to be in line with other implementations, so perhaps Windows really does suck and there's not much we can do about it.

The leading theory. ;-)

Re: The Great Ruby Shootout (July 2010)

#22

Post by headius »

jimbokun wrote: post #15
alnayyir wrote: post #13

Earlier quoted context omitted.

Java != JVM

But invokedynamic is part of Java 7. http://java.sun.com/developer/technicalArticles/DynTypeLang/

InvokeDynamic and other Java 7 features (method handles, NIO2) are definitely going to improve JRuby's situation, but maybe not in the way you expect. Indy and method handles will largely allow us to delete (or not load) code we currently generate to get the same effect. Smaller runtime, possibly a smaller distribution. NIO2 will give us more direct access to streams, process handling, and so on, so we can delete hacks we've written to do all that ourselves.

But perhaps the most interesting aspect of any new major Java release is that they're usually 10-20% faster across the board, due to new and better optimizations. As the saying goes, if your Java app isn't fast enough, upgrade to a newer JVM.

I think invokedynamic combined with more runtime profile-driven optimizations in JRuby could easily double JRuby's Ruby-execution performance (or better), and in many cases reduce memory churn too. Lots of good things coming...it's nice to have an army of VM engineers on your side.

Re: The Great Ruby Shootout (July 2010)

#23

Post by headius »

rbranson wrote: post #3

JRuby would be the future if the startup times weren't so horrendous. It makes it very unpalatable for scripting. Nailgun is not robust enough. I have a feeling this is going to be more of a JVM problem than something Nutter and team is going to tackle. So is the future Rubinius and JRuby?

A large part of startup time is out of our hands, but we continue to look for workarounds.

Most current startup slowness is due to the JVM itself running slow during the first few seconds of execution. Ruby scripts execute as they boot, which means we have to parse and run them. But until the JVM's been up for a little while, nothing in JRuby itself has even JITted to native code...and interpreted JVM bytecode runs even slower than Ruby on most of my tests.

The other complicating factor is the fact that Ruby applications run so much code on boot. RubyGems, for example, degrades startup time O(n) based on how many gems you have installed. Hacks like faster_rubygems (gem install faster_rubygems) help, but changes are needed in RubyGems proper.

In any case, we feel the startup pain too. I've blogged a few tips about startup time here: http://blog.headius.com/2010/03/jruby-startup-time-tips.html, and I've described some of the challenges here: http://blog.headius.com/2010/06/my-short-list-of-key-missing...

We're working on it...we really are. We're just fighting against a decade of JVM folks who never run command-line tools.

Re: The Great Ruby Shootout (July 2010)

#26

Post by headius »

rbranson wrote: post #14
cageface wrote: post #7

Earlier quoted context omitted.

I'm much, much more concerned about the memory usage of JRuby than the startup times. For toss-off scripts I can keep using MRI. I've read that JRuby memory usage scales better than MRI though, so you pay a higher cost upfront but can actually accommodate more workers on the same hardware. I haven't gotten far enough in my exploration of JRuby to see if this is the case in my own apps. Hats off to all the devs workin…

In my tests, it's still worse than MRI. Memory-wise, the best between 1.8.7, 1.9.1, and JRuby 1.5 was still 1.8.7. Of course, if this is the difference between storing stuff in memory and loading from disk, 1.8.7 would be faster by orders of magnitude. The ability to use stable, high-performance JVM libraries trumps this concern in many cases though.

It's also worth pointing out that while a JRuby/Rails instance might take 100-200MB, that's all you need to scale a site across pretty much any number of cores. MRI and REE both need to spin up multiple processes to handle concurrent requests, so very quickly the JRuby memory size becomes a tremendous win (think of 25-50 MRI instances using 20-50MB of memory each...you get the picture).

Re: The Great Ruby Shootout (July 2010)

#27

Post by headius »

messel wrote: post #24

The curve I dream of seeing: c++ slower than any good coverage flavor of Ruby.

I'd like to see that too. In JRuby, we may be able to get Ruby-to-Ruby calls to perform as well as Java calls, which would at least get that bottleneck out of the way. The remaining performance issues, however, are usually the rate at which objects can be allocated. In order to reduce that we may need a little JVM help (escape analysis that works well enough to actually eliminate allocations) and we may start to explore optional static typing, to allow really reducing numeric operations to raw primitive math.

At this point, we realize that sometimes you really do need native performance, and we're not taking any options off the table to get there.

Re: The Great Ruby Shootout (July 2010)

#28

Post by isaacforce »

rbranson wrote: post #3

JRuby would be the future if the startup times weren't so horrendous. It makes it very unpalatable for scripting. Nailgun is not robust enough. I have a feeling this is going to be more of a JVM problem than something Nutter and team is going to tackle. So is the future Rubinius and JRuby?

I have both MRI and JRuby installed on my systems. For quickie scripts I use 'ruby' and for anything with a noticeable runtime (Or multi-core processing!) I use 'jruby'. I'm sure the JRuby fellows will continue working on startup time, but for me at least, it's not an issue.

I think having a Ruby toolbox instead of a single big hammer is a win for everybody.

Re: The Great Ruby Shootout (July 2010)

#29

Post by Confusion »

rbranson wrote: post #3

JRuby would be the future if the startup times weren't so horrendous. It makes it very unpalatable for scripting. Nailgun is not robust enough. I have a feeling this is going to be more of a JVM problem than something Nutter and team is going to tackle. So is the future Rubinius and JRuby?

It's definitely a JVM problem, but contrary to Java, there is a sort of REPL available. Instead if writing a file, modifying it and running 'jruby my-script.rb', just issue 'load myscript.rb' in your (j)irb. That makes for much faster iterations.

Re: The Great Ruby Shootout (July 2010)

#30

Post by rue »

rbranson wrote: post #16
jon_dahl wrote: post #5

Earlier quoted context omitted.

For scripting, that's definitely a problem. But it's not a big deal for long-running processes (e.g. a web application server). But Ruby definitely benefits from both: a fast Ruby with ~zero startup time, and a fast Ruby on the JVM. 3 years ago, Ruby implementation problems became the hot topic at Ruby conferences. Ruby was a beautiful language with an ugly implementation. It's great to see that the community has exe…

While I agree that it's not important for long-running applications, there is still a lot of scripting done in Ruby, and it's very frustrating to get a a 5-to-10 second wait just to tell someone their arguments were invalid. Waiting that long for tests is also such a drag on iteration. I think once JRuby has stabilized the native gem support and has some sort of Passenger-like deployment option (Glassfish and Warbler…

Use MRI for short-lived and JRuby for long-lived. Problem solved?
Post reply on HN