Massive Ruby speed boost with JRuby and Java 7. • Hacker News Board

Re: Massive Ruby speed boost with JRuby and Java 7.

#61

Post by tedunangst » Tue, Dec 20, 2011, 7:01 PM UTC

kennystone wrote: post #58

moe wrote: post #54

I've been eyeing jruby for quite a while but never had much success with it. There seems to always be the odd gem not supporting it (e.g. jruby-openssl is broken which a lot of my stuff depends on).

Invoke dynamic means less of a need for C-Extensions, which is probably one of the largest sources of pain for JRuby compatibility. The JVM is better at running plain old ruby code now, so there's less need to go outside of it with C or Java.

Well, you don't need extensions, but lots of library interfaces still use them because they haven't been updated to FFI. So sometimes you can't use the code you want. The good news is that it's usually pretty painless to whip up your own FFI binding.

Re: Massive Ruby speed boost with JRuby and Java 7.

#62

Post by headius » Tue, Dec 20, 2011, 7:02 PM UTC

pmjordan wrote: post #57

jcromartie wrote: post #29

Earlier quoted context omitted.

Why would respect for the MRI devs stop you from publishing benchmarks? It's not an insult to say that something else is faster. I'd imagine the competition might drive them to improve if at all possible.

I'm not really familiar with the ruby development process and community and haven't used the language much, but I could imagine the MRI developers spend considerable time advancing the actual design of the language (syntax, standard APIs, etc.) whereas the other Ruby implementations "only" need to implement that design and thus can maybe focus more on performance. If MRI were to die, that design work would still need…

Yes, I'd say that's at least partially true. The innovation on JRuby's side is in new ways to integrate with the JVM, rather than in API or language design. MRI folks need to innovate both in how they design the language and APIs and in how they implement them.

Re: Massive Ruby speed boost with JRuby and Java 7.

#63

Post by headius » Tue, Dec 20, 2011, 7:04 PM UTC

moe wrote: post #54

I've been eyeing jruby for quite a while but never had much success with it. There seems to always be the odd gem not supporting it (e.g. jruby-openssl is broken which a lot of my stuff depends on).

OpenSSL is a terribly complex library to support, but we've managed to steadily improve our pure-Java port (originally a herculean effort by Ola Bini, it wraps bouncycastle in an OpenSSL-lookalike API). There's still probably issues, but we have tried to fix them as they come up.

Re: Massive Ruby speed boost with JRuby and Java 7.

#64

Post by headius » Tue, Dec 20, 2011, 7:06 PM UTC

kennystone wrote: post #58

moe wrote: post #54

I've been eyeing jruby for quite a while but never had much success with it. There seems to always be the odd gem not supporting it (e.g. jruby-openssl is broken which a lot of my stuff depends on).

Invoke dynamic means less of a need for C-Extensions, which is probably one of the largest sources of pain for JRuby compatibility. The JVM is better at running plain old ruby code now, so there's less need to go outside of it with C or Java.

That's definitely our hope. JRuby has come further than any implementation in making Ruby fast enough to replace C or Java, and if it's possible to write more in Ruby then all implementations will benefit. C extensions are the number one thing holding the C/++ implementations back.

Re: Massive Ruby speed boost with JRuby and Java 7.

#65

Post by jshen » Tue, Dec 20, 2011, 7:19 PM UTC

maybe I'm a weirdo, but I care a lot more about memory consumption than processing performance. Starting up irb with jruby-head and java 7u2 uses around 80MB. 80MB for vanilla irb!

MRI uses around 8MB

Edit: I get down voted for saying that I care about memory consumption? My use case may not be everyones, but it's a legitimate issue is it not?

Re: Massive Ruby speed boost with JRuby and Java 7.

#66

Post by headius » Tue, Dec 20, 2011, 7:35 PM UTC

tedunangst wrote: post #61

kennystone wrote: post #58

Earlier quoted context omitted.

Invoke dynamic means less of a need for C-Extensions, which is probably one of the largest sources of pain for JRuby compatibility. The JVM is better at running plain old ruby code now, so there's less need to go outside of it with C or Java.

Well, you don't need extensions, but lots of library interfaces still use them because they haven't been updated to FFI. So sometimes you can't use the code you want. The good news is that it's usually pretty painless to whip up your own FFI binding.

We've been very lucky to have a lot of ingenious JRuby users help port some of those extensions or produce API-compatible wrappers around equivalent Java libs. There's still C exts we don't have equivalent JRuby exts for, but things are much better than they used to be.

Re: Massive Ruby speed boost with JRuby and Java 7.

#67

Post by headius » Tue, Dec 20, 2011, 7:38 PM UTC

jshen wrote: post #65

maybe I'm a weirdo, but I care a lot more about memory consumption than processing performance. Starting up irb with jruby-head and java 7u2 uses around 80MB. 80MB for vanilla irb! MRI uses around 8MB Edit: I get down voted for saying that I care about memory consumption? My use case may not be everyones, but it's a legitimate issue is it not?

Memory is cheap?

JRuby largely uses more memory because we have a "real" garbage collector. The JVM, unlike MRI, allocates much more memory than the app needs. That gives it freedom to delay GC runs and push old objects into rarely-GCed sections of memory. It does also mean JRuby uses more memory on startup.

JRuby also has the JVM's subsystems booted and considerably more complex caching and optimization logic. This does increase the base size of a JRuby runtime.

It's possible to get JRuby to start in under 30MB if you explicitly force the JVM to use less memory with -J-Xmx30M or lower. But is it that big a deal? Let the JVM breathe, and your code runs better as a result.

Re: Massive Ruby speed boost with JRuby and Java 7.

#68

Post by headius » Tue, Dec 20, 2011, 7:47 PM UTC

igouy wrote: post #52

headius wrote: post #25

Earlier quoted context omitted.

Oh, and regarding JRuby versus Python 3...yes, if Ruby 1.9 smokes Python, and JRuby is faster than Ruby 1.9, then JRuby should smoke Python even more. I have not done the comparisons myself, though.

> if Ruby 1.9 smokes Python A big IF? :-) http://shootout.alioth.debian.org/u32/benchmark.php?test=all...

I know some of the implementations there use native libraries that are built into python but only available via RubyGems for Ruby, like numpy. That makes it less of a language shootout (at times) and more of a "who ships the best C libs" shootout.

I believe MRI is generally faster than Python now when they're doing roughly equivalent work in Ruby and Python.

Re: Massive Ruby speed boost with JRuby and Java 7.

#69

Post by tlogan » Tue, Dec 20, 2011, 7:52 PM UTC

It is also important to mention that java7 G1 garbage collector gives much better results for certain installations (of course test it before deploying it on production). The pauses are much more predictable and, in my case, much shorter.

Here is how to get Oracle Java7 on Ubuntu:

  wget http://download.oracle.com/otn-pub/java/jdk/7/jdk-7-linux-x64.tar.gz
  tar -xvf jdk-7-linux-x64.tar.gz 
  sudo mkdir -p /usr/lib/jvm/      
  sudo mv jdk1.7.0/ /usr/lib/jvm/
  sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0/jre/bin/java 0
  sudo update-alternatives --config java

Re: Massive Ruby speed boost with JRuby and Java 7.

#70

Post by jshen » Tue, Dec 20, 2011, 7:56 PM UTC

headius wrote: post #67

jshen wrote: post #65

maybe I'm a weirdo, but I care a lot more about memory consumption than processing performance. Starting up irb with jruby-head and java 7u2 uses around 80MB. 80MB for vanilla irb! MRI uses around 8MB Edit: I get down voted for saying that I care about memory consumption? My use case may not be everyones, but it's a legitimate issue is it not?

Memory is cheap? JRuby largely uses more memory because we have a "real" garbage collector. The JVM, unlike MRI, allocates much more memory than the app needs. That gives it freedom to delay GC runs and push old objects into rarely-GCed sections of memory. It does also mean JRuby uses more memory on startup. JRuby also has the JVM's subsystems booted and considerably more complex caching and optimization logic. This…

First, I'd like to say that I use JRuby a lot, and you guys have done an amazing job on it. I complain because I like JRuby, and wish I could use it more :)

I did set the Xmx and it still uses around 80MB. I'm not sure what in the world the JVM is doing there, but the permgen is another 20MB or so and then other native stuff, thread overhead, etc.

Is memory cheap? Yes and no. At work we have a rails site with two REST api backends both ruby. I also love RubyMine. Now here's my problem, I can't run all of those on the JVM on my macbook air. 4 gigs is not enough if I also have a browser open!

This is not a problem at all if I switch the apps to MRI and switch my editor to a non JVM based editor. I think the JVM people need to recognize this problem.