Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Great Ruby Shootout (July 2010)

programmingzen.com

Post reply on HN
Print view View original

31–40 of 45 posts

Re: The Great Ruby Shootout (July 2010)

#31

Post by cageface »

headius wrote: post #26
rbranson wrote: post #14

Earlier quoted context omitted.

In my tests, it's still worse than MRI. Memory-wise, the best between 1.8.7, 1.9.1, and JRuby 1.5 was still 1.8.7. Of course, if this is the difference between storing stuff in memory and loading from disk, 1.8.7 would be faster by orders of magnitude. The ability to use stable, high-performance JVM libraries trumps this concern in many cases though.

It's also worth pointing out that while a JRuby/Rails instance might take 100-200MB, that's all you need to scale a site across pretty much any number of cores. MRI and REE both need to spin up multiple processes to handle concurrent requests, so very quickly the JRuby memory size becomes a tremendous win (think of 25-50 MRI instances using 20-50MB of memory each...you get the picture).

Yeah that's what I'm hoping to see as I transition a couple of test apps over. The scala/lift folks see this as a big advantage for their JVM web stack too.

Re: The Great Ruby Shootout (July 2010)

#32

Post by headius »

isaacforce wrote: post #28
rbranson wrote: post #3

JRuby would be the future if the startup times weren't so horrendous. It makes it very unpalatable for scripting. Nailgun is not robust enough. I have a feeling this is going to be more of a JVM problem than something Nutter and team is going to tackle. So is the future Rubinius and JRuby?

I have both MRI and JRuby installed on my systems. For quickie scripts I use 'ruby' and for anything with a noticeable runtime (Or multi-core processing!) I use 'jruby'. I'm sure the JRuby fellows will continue working on startup time, but for me at least, it's not an issue. I think having a Ruby toolbox instead of a single big hammer is a win for everybody.

We hear this a lot. Hopefully we can keep improving startup times, but when it's possible to use either Ruby or JRuby, it makes a pretty good combination.

Re: The Great Ruby Shootout (July 2010)

#33

Post by rbranson »

headius wrote: post #26
rbranson wrote: post #14

Earlier quoted context omitted.

In my tests, it's still worse than MRI. Memory-wise, the best between 1.8.7, 1.9.1, and JRuby 1.5 was still 1.8.7. Of course, if this is the difference between storing stuff in memory and loading from disk, 1.8.7 would be faster by orders of magnitude. The ability to use stable, high-performance JVM libraries trumps this concern in many cases though.

It's also worth pointing out that while a JRuby/Rails instance might take 100-200MB, that's all you need to scale a site across pretty much any number of cores. MRI and REE both need to spin up multiple processes to handle concurrent requests, so very quickly the JRuby memory size becomes a tremendous win (think of 25-50 MRI instances using 20-50MB of memory each...you get the picture).

Yeah I see that. My tests primarily revolved around single-threaded event driven code, and retaining large container objects (millions of objects). MRI was much more efficient. I understand that there's a lot more going on under the hood in JRuby just to make this possible, but for this use case, the result is what mattered. Although we ended up using Tokyo Tyrant (with a hack to pre-disk-cache all the data) anyway.

Re: The Great Ruby Shootout (July 2010)

#35

Post by acangiano »

leif wrote: post #34

Means and (especially) medians mean little to nothing when you're doing them over entirely different benchmarks. Why bother? A 3D plot would be a lot clearer.

> Means and (especially) medians mean little to nothing

It's the other way around. Given the skewed nature of the data, the median is a better measure of the central tendency of the data set.

But note that I'm not plotting the median only. I use a box plot which gives you a much better statistical picture than using a simple median (or mean). Nevertheless, summarizing different benchmarks has its limitations and box plots give you a rough, general idea of what's going on.

Re: The Great Ruby Shootout (July 2010)

#36

Post by donw »

headius wrote: post #19

I can comment on the JRuby results. For the base perf numbers, I'm not surprised. We've known that we're roughly on par with 1.9.2 for some time, and many of the benchmarks in question have started to reach a point of irreducible complexity (e.g. you can only slice and dice strings so fast). It's good to see JRuby remains at or near the front of the pack as far as performance, especially considering we've made no maj…

Tooting my own metaphorical horn a bit, but it was relatively easy to put together a Jetty-Rack interface for high-performance webapps. It's called Mizuno, and lives at http://github.com/matadon/mizuno

Internally it uses Jetty's event-driven I/O, so performance is on a par with Thin and Passenger, at least on my MacBook.

I'm a bit pressed for time at the moment, so if anybody wants to add in the cometd servlet (it's in the repos) and write a tutorial, that'd be awesome, and if not, I'll get to it in a few weeks.

Re: The Great Ruby Shootout (July 2010)

#37

Post by xpaulbettsx »

acangiano wrote: post #21
headius wrote: post #19

I can comment on the JRuby results. For the base perf numbers, I'm not surprised. We've known that we're roughly on par with 1.9.2 for some time, and many of the benchmarks in question have started to reach a point of irreducible complexity (e.g. you can only slice and dice strings so fast). It's good to see JRuby remains at or near the front of the pack as far as performance, especially considering we've made no maj…

> The Linux versus Windows numbers are a bit surprising to me. If I were to make a guess, I'd guess that the JVMs used were not identical (perhaps like Isaac Gouy mentions, one platform was 64-bit and the other wasn't) or some other detail altered the performance characteristics of the test. The same identical version was used: Java HotSpot(TM) 64-Bit Server VM 1.6.0_20. > But the performance drop does seem to be in…

I don't know that this theory totally holds water (Windows OS developer here, announcing obvious bias) - looking at the results, the one place Windows really gets nailed is the I/O test; I suspect there's some significant optimizations that could be done there, as I/O on Windows OS in general certainly isn't 4x slower than Linux.

Re: The Great Ruby Shootout (July 2010)

#38

Post by bodhi »

acangiano wrote: post #35
leif wrote: post #34

Means and (especially) medians mean little to nothing when you're doing them over entirely different benchmarks. Why bother? A 3D plot would be a lot clearer.

> Means and (especially) medians mean little to nothing It's the other way around. Given the skewed nature of the data, the median is a better measure of the central tendency of the data set. But note that I'm not plotting the median only. I use a box plot which gives you a much better statistical picture than using a simple median (or mean). Nevertheless, summarizing different benchmarks has its limitations and box…

Did you normalise the results before plotting them and calculating means and medians? Or is it a summary of the raw data? I could probably find out myself, but since you are posting here, I thought I'd ask.

Re: The Great Ruby Shootout (July 2010)

#39

Post by necubi »

headius wrote: post #19

I can comment on the JRuby results. For the base perf numbers, I'm not surprised. We've known that we're roughly on par with 1.9.2 for some time, and many of the benchmarks in question have started to reach a point of irreducible complexity (e.g. you can only slice and dice strings so fast). It's good to see JRuby remains at or near the front of the pack as far as performance, especially considering we've made no maj…

I'm very excited to see C-extension support. That might finally convince me to port some of my projects.

Re: The Great Ruby Shootout (July 2010)

#40

Post by headius »

rbranson wrote: post #33
headius wrote: post #26

Earlier quoted context omitted.

It's also worth pointing out that while a JRuby/Rails instance might take 100-200MB, that's all you need to scale a site across pretty much any number of cores. MRI and REE both need to spin up multiple processes to handle concurrent requests, so very quickly the JRuby memory size becomes a tremendous win (think of 25-50 MRI instances using 20-50MB of memory each...you get the picture).

Yeah I see that. My tests primarily revolved around single-threaded event driven code, and retaining large container objects (millions of objects). MRI was much more efficient. I understand that there's a lot more going on under the hood in JRuby just to make this possible, but for this use case, the result is what mattered. Although we ended up using Tokyo Tyrant (with a hack to pre-disk-cache all the data) anyway.

Object sizes in JRuby are certainly larger, especially if you run on a 64-bit JVM which necessarily has 64-bit reference fields for all object references (basically everything in Ruby). It doesn't surprise me at all to see a single-threaded case with a lot of objects use more memory, but it would be interesting to see how much of that was unused heap space and how much was actually live data. MRI's less-efficient conservative collector can live in a smaller memory space, but you sacrifice performance.
Post reply on HN