Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New Rust hash table leads Benchmarks Game

benchmarksgame.alioth.debian.org

Post reply on HN
Print view View original

121–130 of 156 posts

Re: New Rust hash table leads Benchmarks Game

#121

Post by igouy »

saurik wrote: post #120
nisa wrote: post #118

Earlier quoted context omitted.

> it.unimi.dsi.fastutil.longs.Long2IntOpenHashMap?!? That's just a library that provides data structures without boxing of int,double... in Java. This eliminates a lot of overhead.

Oh, I absolutely understand why it is interesting and fast: what I don't understand is how it satisfies the rules, as it is effectively "some random project with a hashtable". Is this particular one so famous that it should be allowed instead of java.util.HashMap? Can I just publish my C++ hashtable and then rely on it, in which case the rule makes no sense? That's why I tack on "?!".

> instead of java.util.HashMap

Not "instead of" as-well-as.

Re: New Rust hash table leads Benchmarks Game

#122

Post by saurik »

igouy wrote: post #121
saurik wrote: post #120

Earlier quoted context omitted.

Oh, I absolutely understand why it is interesting and fast: what I don't understand is how it satisfies the rules, as it is effectively "some random project with a hashtable". Is this particular one so famous that it should be allowed instead of java.util.HashMap? Can I just publish my C++ hashtable and then rely on it, in which case the rule makes no sense? That's why I tack on "?!".

> instead of java.util.HashMap Not "instead of" as-well-as.

This entire subthread is clearly a digression ;p, but why do you say that? The rules were about using a built-in or standard collection: this Java implementation does not use HashMap and instead uses a random project with a better data structure for this use case. This is a loophole to bypass the restriction on writing your own hashtable: you just have to publish it first? ;P

Re: New Rust hash table leads Benchmarks Game

#123

Post by gigatexal »

Etzos wrote: post #18
gigatexal wrote: post #13

The rust version is using multiple cpus using a pool concept (which looks a lot like the multiprocessing module from python so kudos there). But the C version is single threaded from what I can tell. So rust is safe but threaded to be faster than single threaded C which isn't that much slower. Hmm...

From what I can see the C version is using OpenMP and uses all of the available CPU cores.

Is the C version optimized? Could you conclude that parallelism in rust is faster/ better than C + openMP in this case considering rust is to replace C++ and is safe at least in this submission?

Re: New Rust hash table leads Benchmarks Game

#124

Post by saurik »

igouy wrote: post #119
saurik wrote: post #114

> Some language implementations have hash tables built-in; some provide a hash table as part of a collections library; some use a third-party hash table library. (For example, use either khash or CK_HT for C language k-nucleotide programs.) The hash table algorithm implemented is likely to be different in different libraries. > Please don't implement your own custom "hash table" - it will not be accepted. > The work…

> If I were to do this in C++ … I would use the C++11 std::unordered_map … http://benchmarksgame.alioth.debian.org/play.html#contribute

The thesis of my comment was that we should be surprised that any language does poorly on this benchmark, particularly ones that have similar kinds of targeting, and that if we cared about this benchmark (and I claim we don't), we should all pitch in, possibly upstream to fix various languages and their standard libraries to nail this benchmark. However, I also believe the rules of this benchmark are awkward and even flawed, and that it isn't clear to me that it is worth anyone's time to do that.

I am not lamenting that someone should spend more time on this: I am lamenting that tons of people seem to care about it at all, it is not a "microbenchmark" (as some are calling it), and I think the main lesson we can learn from it is "there is something subtely wrong, either with the implementation that was contributed for this benchmark, the language's runtime, or it's standard library", as given these rules we really should expect every language to be similarly in performance.

And so, if we all cared about this benchmark, I bet we could figure out what is going on and get every open source language down under 25s. Past that point, I think the rules are such that this isn't even a fun game much less a useful metric of anything worth measuring, and you are probably wasting your time contributing. I guess, to make this subthread go somewhere: why do you disagree?

Re: New Rust hash table leads Benchmarks Game

#125

Post by igouy »

saurik wrote: post #122
igouy wrote: post #121

Earlier quoted context omitted.

> instead of java.util.HashMap Not "instead of" as-well-as.

This entire subthread is clearly a digression ;p, but why do you say that? The rules were about using a built-in or standard collection: this Java implementation does not use HashMap and instead uses a random project with a better data structure for this use case. This is a loophole to bypass the restriction on writing your own hashtable: you just have to publish it first? ;P

Java #6 => HashMap

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Java #3 => HashMap

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Java #5 => HashMap

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Java #4 => HashMap

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Re: New Rust hash table leads Benchmarks Game

#126

Post by igouy »

saurik wrote: post #124
igouy wrote: post #119

Earlier quoted context omitted.

> If I were to do this in C++ … I would use the C++11 std::unordered_map … http://benchmarksgame.alioth.debian.org/play.html#contribute

The thesis of my comment was that we should be surprised that any language does poorly on this benchmark, particularly ones that have similar kinds of targeting, and that if we cared about this benchmark (and I claim we don't), we should all pitch in, possibly upstream to fix various languages and their standard libraries to nail this benchmark. However, I also believe the rules of this benchmark are awkward and even…

> it is not a "microbenchmark"

Home page -- "Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"

http://benchmarksgame.alioth.debian.org/

Re: New Rust hash table leads Benchmarks Game

#127

Post by rbehrends »

flukus wrote: post #112
rbehrends wrote: post #102

Earlier quoted context omitted.

> The part where Java really falls down is in memory use & management, which you can see on the binary-tree & mandelbrot benchmarks, where it's roughly 4x slower than C. binary-tree is not useful for comparing GCed and non-GCed languages. For non-GCed languages, you are allowed to use a memory pool of your choice (the C version uses the Apache Portable Runtime library), for GCed languages you are required to use the…

> For non-GCed languages, you are allowed to use a memory pool of your choice (the C version uses the Apache Portable Runtime library), for GCed languages you are required to use the standard GC with the default settings (no adjustment of GC parameters permitted). This is apples and oranges. Doesn't that match with how a library would be used in the real world? A c library can create it's own memory pool but a GC one…

If I were to run a performance-critical application, I'd definitely tune the GC accordingly. It's why the JVM offers several garbage collectors in the first place, for example.

Also, GCed languages aren't prevented from using memory pools, but often they are not part of their common libraries, because there's less need for them.

Re: New Rust hash table leads Benchmarks Game

#128

Post by acqq »

mbrubeck wrote: post #58

In other Rust/benchmarksgame news, I just submitted a simple fix to the Rust program for "reverse-complement" that makes it faster than the fastest C++ program, on my computer. The old version was spending 2/3 of its time just reading the input into memory, because it wasn't allocating a large enough buffer up front. https://github.com/TeXitoi/benchmarksgame-rs/pull/44 I'm also working on some additional changes that…

Thanks!

I like Rust for what it does in advancing the state of the art of the languages, but I also like how this example demonstrates how hard it is to avoid "unsafe" constructs and remain competitive.

https://github.com/TeXitoi/benchmarksgame-rs/blob/master/src...

Re: New Rust hash table leads Benchmarks Game

#129

Post by Etzos »

cnnsucks wrote: post #115
stcredzero wrote: post #79

Java is showing quite impressive numbers! 50% overhead over native C implementations was often cited as a good guess for the ultimate efficiency of JIT code generation back in the Self Hotspot days. People who were trying to castigate Go early on as having "Java-like speeds" were really just showing their ignorance of the state of the art of JIT compilation for managed languages and the JVM. Such outdated folk knowle…

>> Java is showing quite impressive numbers! 359% more RAM isn't very impressive. Even less so when one considers the 20+ years of effort spent to achieve it. You know what impresses me? A 22 month old language besting everything else while guaranteeing no segfaults or NPEs at compile time. That's impressive.

Not that I don't disagree with the general idea of this post, but it's worth pointing out that while Rust is fairly young based on the 1.0 release date, there was a large amount of time prior to that where the language underwent a number of changes.

It's also worth remembering that this is just a game, and as such it's somewhat of an apples-to-oranges comparison. Java's RAM usage may not be impressive compared to C, but that doesn't make the results overall any less impressive, especially knowing what it was like before those 20+ years of effort.

I don't see a reason that both Rust and Java's results can't be impressive in their own right. Java's numbers are (for the most part) impressive compared to C and Rust's numbers are also impressive compared to C, just in a different way.

Re: New Rust hash table leads Benchmarks Game

#130

Post by saurik »

igouy wrote: post #125
saurik wrote: post #122

Earlier quoted context omitted.

This entire subthread is clearly a digression ;p, but why do you say that? The rules were about using a built-in or standard collection: this Java implementation does not use HashMap and instead uses a random project with a better data structure for this use case. This is a loophole to bypass the restriction on writing your own hashtable: you just have to publish it first? ;P

Java #6 => HashMap http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... Java #3 => HashMap http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... Java #5 => HashMap http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... Java #4 => HashMap http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Those are different implementations; I know those exist, and which is why I had specifically said there were multiple implementations and pointed at the "faster one", which is obviously the one most people are going to be paying attention to, and which is the one that is also most relevant as we could and probably should expect the implementations for other languages to use some crazy one-off library: it demonstrates and undermines the limitations placed on the various languages seem arbitrary enough that you can't read into these results as being indicative of the languages themselves, but simultaneously that we shouldn't even care as the fast Java entry effectively "cheated".

I am going to take a step back: you seem irked by my comments, due to the knee-jerk link to have me contribute a different implementation for C++ as well as arguing with the short reply what I really maintain is a nearly-offtopic point about the Java library in question here (both of which I am reading as slightly aggressive towards me or my comment).

Do you really like this benchmark? Are you super excited that Rust is slightly faster right now than C/C++/Java? What is going through your head when you read my comments? Do you disagree that we should expect all native-ish languages to do equally-enough well for most people given these constraints? Is it that my comment is coming off to you as "no fun allowed", as I am trying to point out that this is a meaningless competition that mostly teaches us about different things than "who is winning"?

In fact, looking at your recent comment history, I am realizing you are being super aggressive about this with everyone, and are nigh unto spamming the play.html link to everyone who tries to comment about the high-level idea of "what are we doing here". What's up?

Post reply on HN