Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Announcing Topaz: A New Ruby

docs.topazruby.com

Post reply on HN
Print view View original

181–190 of 190 posts

Re: Announcing Topaz: A New Ruby

#181

Post by Cowen »

callum85 wrote: post #122
Cowen wrote: post #67

Earlier quoted context omitted.

Whenever I see frontpages for these kinds of projects like "a faster X" or "X written in Blub", the first thing I want to see on the frontpage is how this new project compares to X in terms of quality and performance. Even specious benchmarks would help more than zero benchmarks. I wish more frontpages for these kinds of projects would do that.

"Even specious benchmarks would help more than zero benchmarks." I disagree. Zero benchmarks is definitely better than specious benchmarks.

To clarify, I was trying to use "specious" as a synonym of "flawed." I thought this was the common usage, but apparently not.

As to your point, obviously no one should be making any decisions off of flawed benchmarks, but flawed benchmarks (not so far as outright lies, just flawed) at least give me an objective justification to investigate further.

Even some flawed benchmarks could help turn the initial tide of responses like "this is X written in Blub, it's bound to be better!" or "this is a faster X! Now everything will be twice as fast!" They're silly examples, but it seems like every time a new technology comes out, these are the kinds of knee-jerk, overly-optimistic reactions people tend to have.

Re: Announcing Topaz: A New Ruby

#182

Post by kenko »

Cowen wrote: post #181
callum85 wrote: post #122

Earlier quoted context omitted.

"Even specious benchmarks would help more than zero benchmarks." I disagree. Zero benchmarks is definitely better than specious benchmarks.

To clarify, I was trying to use "specious" as a synonym of "flawed." I thought this was the common usage, but apparently not. As to your point, obviously no one should be making any decisions off of flawed benchmarks, but flawed benchmarks (not so far as outright lies, just flawed) at least give me an objective justification to investigate further. Even some flawed benchmarks could help turn the initial tide of respo…

" To clarify, I was trying to use "specious" as a synonym of "flawed." I thought this was the common usage, but apparently not."

"Specious" does mean, more or less, flawed.

Zero benchmarks are better than flawed benchmarks.

Re: Announcing Topaz: A New Ruby

#183

Post by axomhacker »

snissn wrote: post #108
Pewpewarrows wrote: post #10

Completely unscientific, but if these outputs are any indication, this is going to be great news for Rubby users in the future... $ time ruby -e "puts 'hello world'" hello world real 0m0.184s user 0m0.079s sys 0m0.092s $ time ~/Downloads/topaz/bin/topaz -e "puts 'hello world'" hello world real 0m0.007s user 0m0.002s sys 0m0.004s

$ time ruby -e "puts 'hello world'" The program 'ruby' can be found in the following packages: * ruby1.8 * ruby1.9.1 Ask your administrator to install one of them real 0m0.060s user 0m0.040s sys 0m0.016s

Don't know if you are trolling. But if this is genuine: this is rbenv telling you that you have multiple rubies installed.

1. Pick one (just for this session): $ rbenv shell ruby1.9.1

2. And then run the example.

By the way 1.9.1 is really old already, 1.9.3 has a lot more bug fixes.

Re: Announcing Topaz: A New Ruby

#184

Post by alrs »

axomhacker wrote: post #183
snissn wrote: post #108

Earlier quoted context omitted.

$ time ruby -e "puts 'hello world'" The program 'ruby' can be found in the following packages: * ruby1.8 * ruby1.9.1 Ask your administrator to install one of them real 0m0.060s user 0m0.040s sys 0m0.016s

Don't know if you are trolling. But if this is genuine: this is rbenv telling you that you have multiple rubies installed. 1. Pick one (just for this session): $ rbenv shell ruby1.9.1 2. And then run the example. By the way 1.9.1 is really old already, 1.9.3 has a lot more bug fixes.

ruby1.9.1 on Debian isn't ruby 1.9.1.

The Ruby language changed between 1.9 and 1.9.1, so a new package name had to be created.

If 1.9.1 was just called "1.9" it would break all of the packages in Debian that depend on whatever language features were different between 1.9 and 1.9.1.

"ruby1.9.1" in Debian 7.0 provides version 1.9.3.194.

Re: Announcing Topaz: A New Ruby

#185

Post by nessus42 »

kenko wrote: post #182
Cowen wrote: post #181

Earlier quoted context omitted.

To clarify, I was trying to use "specious" as a synonym of "flawed." I thought this was the common usage, but apparently not. As to your point, obviously no one should be making any decisions off of flawed benchmarks, but flawed benchmarks (not so far as outright lies, just flawed) at least give me an objective justification to investigate further. Even some flawed benchmarks could help turn the initial tide of respo…

" To clarify, I was trying to use "specious" as a synonym of "flawed." I thought this was the common usage, but apparently not." "Specious" does mean, more or less, flawed. Zero benchmarks are better than flawed benchmarks.

All benchmarks are flawed in one way or another.

Re: Announcing Topaz: A New Ruby

#186

Post by nirvdrum »

__david__ wrote: post #146
nirvdrum wrote: post #94

Earlier quoted context omitted.

If the goal is to benchmark Ruby execution time, it's unfair. The slow startup time is a valid concern when considering short-lived sessions, but it's kind of meaningless when trying to benchmark a Ruby implementation.

It's not meaningless if you're trying to make a command line app and not a Rails app. Startup time does matter for command line apps (a lot!).

Agreed. But then you're not benchmarking Ruby execution speed, which really makes comparing the two not very worthwhile. By the same notion, then when you compare on something like JRuby, you really should be comparing JDK 6, 7, and 8 builds, along with various startup flags, both JVM and JRuby. E.g., short runs may benefit greatly from turning off JRuby JIT, JAR verification, and using tiered compilation modes. May as well add Nailgun and drip in there as well, since both can speed up startup time on subsequent runs. Which one of these is going to be the JRuby you use in your comparisons? Or are you really going to show 20 different configurations?

You can run into the same problem with MRI and its GC settings. If too low for your test, you're going to hit GC hard. It's best to normalize that out so you have an even comparison. Confounding variables and all that.

There's a lot you can do outside the Ruby container to influence startup time. When comparing two implementations, the defaults are certainly something to consider, but not when trying to see which actually executes Ruby faster. They are two different metrics of performance and should be compared in isolation.

Re: Announcing Topaz: A New Ruby

#187

Post by axomhacker »

alrs wrote: post #184
axomhacker wrote: post #183

Earlier quoted context omitted.

Don't know if you are trolling. But if this is genuine: this is rbenv telling you that you have multiple rubies installed. 1. Pick one (just for this session): $ rbenv shell ruby1.9.1 2. And then run the example. By the way 1.9.1 is really old already, 1.9.3 has a lot more bug fixes.

ruby1.9.1 on Debian isn't ruby 1.9.1. The Ruby language changed between 1.9 and 1.9.1, so a new package name had to be created. If 1.9.1 was just called "1.9" it would break all of the packages in Debian that depend on whatever language features were different between 1.9 and 1.9.1. "ruby1.9.1" in Debian 7.0 provides version 1.9.3.194.

The OP's output looked more like rbenv's output though, right? How did you figure that to be debian's message?

Re: Announcing Topaz: A New Ruby

#188

Post by grn »

fernandezpablo wrote: post #105
grn wrote: post #80

Earlier quoted context omitted.

In order to eliminate the potential bias of startup time I run similar test in many iterations. Here's what I got: $ time ruby -e "10000.times { puts 'hello world' }" > /dev/null real 0m0.102s user 0m0.096s sys 0m0.005s and $ time ./topaz -e "10000.times { puts 'hello world' }" > /dev/null real 0m0.098s user 0m0.071s sys 0m0.026s Any idea why I don't see such big difference?

what is your 'ruby'?

ruby that comes with Debian Squeeze.

    $ ruby --version
    ruby 1.8.7 (2010-08-16 patchlevel 302) [i486-linux]

Re: Announcing Topaz: A New Ruby

#189

Post by scott_s »

nirvdrum wrote: post #149
scott_s wrote: post #148

Earlier quoted context omitted.

The issue is that, to some, "ruby execution time" may include the startup time.

But if it's not executing ruby, then it can't be ruby execution time... That's the point I'm making. By all means, if you want to measure start-up time, that's valid as well, just not for "how fast does this execute Ruby."

You may have a finer-grain definition of "execute Ruby" than someone else.

Re: Announcing Topaz: A New Ruby

#190

Post by Tobu »

simula67 wrote: post #177
fijal wrote: post #18

Earlier quoted context omitted.

PyPy is totally not like LLVM in many regards. It targets a very different demographics. Some differences: * RPython comes with a good garbage collector * the language where you specify what's going on is RPython in which you write an interpreter. Then you get a JIT using a few hints. This is difference than "interpreter in C + compiler to LLVM" scenario by quite a bit. * RPython comes with a set of data structures t…

So its closer to JVM than LLVM ( http://stackoverflow.com/questions/454720/what-are-the-diffe... )?

Not really. The LLVM analogy does capture the toolchain aspect; the RPython toolchain generates a JITted VM from an interpreter description.
Post reply on HN