Rust is now overall faster than C in benchmarks • Hacker News Board

Re: Rust is now overall faster than C in benchmarks

#201

Post by littlestymaar » Sun, Jan 03, 2021, 11:34 PM UTC

pornel wrote: post #80

FartyMcFarter wrote: post #64

Looking at the reverse-complement code, it appears that the Rust and C implementations are using different algorithms: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/... On a quick inspection: - The Rust code is about twice as long. - The Rust code has CPU feature detection and SSE intrinsics, while the C code is more idiomatic. - The lookup…

Nothing stops someone from copying and submitting other implementation's algorithm. There are multiple implementations of each benchmark for every language: • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It's possible that someone has already submitted both algorithms…

> Nothing stops someone

Well, it looks like the submission process[0] and the maintainer[1] do, actually.

[0]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov... [1]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov...

Re: Rust is now overall faster than C in benchmarks

#202

Post by brundolf » Sun, Jan 03, 2021, 11:36 PM UTC

lambda wrote: post #191

steveklabnik wrote: post #167

Earlier quoted context omitted.

Don't know why you're downvoted, this is a great comment. By the way, YARV has been "ruby" since 2007; these benchmarks were run with ruby 3.0.0preview1. The name has probably just never been changed.

Ah, thanks. It's been a while since I've used Ruby seriously. Still not quite sure why "Matz's ruby" is separate on the Benchmark Game rankings, while all other languages are grouped based on the best implementation of the language. Anyhow, that's pretty much irrelevant to my overall point. Thanks for the kind words, not sure why it's getting downvoted other than that I called someone out on their lazy dismissal. But…

Your tone at certain points was overly incendiary, imo (the comment you were replying to was a little bit too). But there was also lots of good substance around those bits. Just my two cents.

Re: Rust is now overall faster than C in benchmarks

#203

Post by burnthrow » Sun, Jan 03, 2021, 11:37 PM UTC

agumonkey wrote: post #5

albertzeyer wrote: post #3

Interestingly, C++ seems to be the fastest overall.

After the rust blossom storm I didn't track cpp implementation evolutions.. did cpp compiler perf/libs increased or was is simply faster and still is the same ?

Was faster and is the same, checking isn't free. Not that Rust is slow. C++ is unsafe by default whereas in Rust it is opt-in (lexically scoped). The Rust implementations don't appear to use `unsafe`.

Re: Rust is now overall faster than C in benchmarks

#204

Post by littlestymaar » Sun, Jan 03, 2021, 11:46 PM UTC

dan-robertson wrote: post #197

Matthias247 wrote: post #93

Apart from those benchmark games a lot of real world C is a lot less performant than people think it might be. I spent a fair amount of time reviewing C code in the last 5 years - and things that pop up in nearly every review are costly string operations. Linear counts due to the use of null terminated strings and extra allocations for substrings to attach null terminators, or just deep copies because ownership can’t…

The reason that C programs often perform well is that it’s so incredibly hard to do anything at all in C (especially something reliable) that one can usually only do the simplest thing possible and this typically means simple data structures, simple algorithms and arrays. In many ways, modern CPUs are particularly designed to run the machine code generated by C compilers on typical C code like this. Pointers and memo…

> The reason that C programs often don’t perform as well as an equivalent rust program[1] is that it’s so incredibly hard to do anything at all in C (especially something reliable) that one can usually only do the simplest thing possible and this typically means simple data structures, simple algorithms and arrays

Brian Cantrill talks[1] about exactly this: in his C version he was using AVL trees because they are easy to implement, while his Rust version was using B-trees just because he could. And as a result, his naive first attempt in Rust outperformed his long-optimized C code.

[1]: https://www.youtube.com/watch?v=HgtRAbE1nBM&t=43m15s

Re: Rust is now overall faster than C in benchmarks

#205

Post by makapuf » Sun, Jan 03, 2021, 11:47 PM UTC

tlb wrote: post #34

skohan wrote: post #24

Earlier quoted context omitted.

How does c++ manage to win here? I'm not doubting, just curious

C++ can inline the comparison function (which is often just a few instructions) at compile time. It can also statically optimize for the stride, using shifts or addressing modes instead of multiplies. C can't do either with qsort, which takes the stride and comparison function pointer as arguments.

Unless comparison function is small and can be inlined, which would trigger the same stride shift optimizations.

Re: Rust is now overall faster than C in benchmarks

#206

Post by josephg » Sun, Jan 03, 2021, 11:50 PM UTC

vbezhenar wrote: post #153

frutiger wrote: post #149

Earlier quoted context omitted.

Mostly agree with your comment but linear search through arrays of size less than a few hundred will typically beat more sophisticated structures such as red-black trees or hashtables. This is due to prefetching and avoidance of unpredictable pointer traversals. Asymptotic complexity is only that: asymptotic. In many programs in many domains the sizes of these data structures will rarely exceed this limit.

It might be fast, but it'll load CPU caches with that data and it'll evict another useful data. Which means that while this particular code will be fast or at least not very slow, some other code will be slow because its data have to be fetched again. I have no idea whether that matters or even easy to measure...

> I have no idea whether that matters or even easy to measure...

It is reasonably easy to measure, and the GP is about right. I've measured a crossover point of around a few hundred items too. (Though I'm sure it'll vary depending on use case and whatnot.)

I made a rope data structure a few years ago in C. Its a fancy string data structure which supports inserts and deletes of characters at arbitrary offsets. (Designed for text editors). The implementation uses a skip list (which performs similarly to a b-tree). At every node we store an array of characters. To insert or delete, we traverse the structure to find the node at the requested offset, then (usually) memmove a bunch of characters at that node.

Q: How large should that per-node array be? A small number would put more burden on the skip list structure and the allocator, and incur more cache misses. A large number will be linearly slower because of all the time spent in memmove.

Benchmarking shows the ideal number is in the ballpark of 100-200, depending on CPU and some specifics of the benchmark itself. Cache misses are extremely expensive. Storing only a single character at each node (like the SGI C++ rope structure does) makes it run several times slower. (!!)

Code: https://github.com/josephg/librope

This is the constant to change if you want to experiment yourself:

https://github.com/josephg/librope/blob/81e1938e45561b0856d4...

In my opinion, hash tables, btrees and the like in the standard library should probably swap to flat lists internally when the number of items in the collection is small. I'm surprised more libraries don't do that.

Re: Rust is now overall faster than C in benchmarks

#207

Post by roca » Sun, Jan 03, 2021, 11:51 PM UTC

littlestymaar wrote: post #195

throwaway894345 wrote: post #179

Earlier quoted context omitted.

I agree with this. Benchmark code differs from real code in in that it approximates the performance ceiling for a language implementation; it's not "ordinary code" or even "somewhat optimized" but usually the most optimal code one can conceive of with little respect paid to competing concerns, like maintainability. Rust aspires to make idiomatic, maintainable code almost as performant as benchmark code by way of zero…

> If C# and Go preclude 95% of the errors found in Python and JS Well Go and C#[1] still suffer from the billion dollar mistake (null pointers), which represents at least 1/3 of errors I've witnessed in JavaScript code, so I'd say they at best removes 70% of errors. And there's also logic errors, for which neither Go's or C#'s type system helps either, so maybe we're at 50% error reductions with Go and C# compared to…

Newtypes are also pretty great for detecting logic errors. (You can use them in Go and C# but they're not idiomatic and usually not zero-cost so people usually don't.)

Re: Rust is now overall faster than C in benchmarks

#208

Post by roca » Sun, Jan 03, 2021, 11:56 PM UTC

throwaway894345 wrote: post #179

jerf wrote: post #125

Earlier quoted context omitted.

I've been fairly convinced for a while that once Rust matures (which is probably fairly close to "now", but I've held this opinion for years) that it's going to have a performance advantage in real code that's going to be hard to capture in benchmarks, because it's easy in a small benchmark to be very careful and ensure that you don't have aliasing, avoid extra copies, etc. Where I expect Rust to really shine perform…

I agree with this. Benchmark code differs from real code in in that it approximates the performance ceiling for a language implementation; it's not "ordinary code" or even "somewhat optimized" but usually the most optimal code one can conceive of with little respect paid to competing concerns, like maintainability. Rust aspires to make idiomatic, maintainable code almost as performant as benchmark code by way of zero…

The optimal point on the tradeoff between developer velocity and performance/correctness depends a lot on the domain, in particular:

-- How much usage do you expect to have? The more usage, the more important performance and correctness are.

-- How critical is your application? The more critical it is, the more important correctness is.

Also, the argument that you can spend developer time to increase correctness just by writing more tests works up to a point, but then it doesn't, because of diminishing returns. With Rust you can eliminate certain entire classes of bugs which tests will never reach.

Re: Rust is now overall faster than C in benchmarks

#209

Post by tutfbhuf » Mon, Jan 04, 2021, 12:01 AM UTC

brundolf wrote: post #199

tutfbhuf wrote: post #196

Nodejs is incredible fast for a interpreted language. It is only ~4 times slower than Rust and only a bit slower than Go or Java (compiled GC languages) in the benchmarks. Compare that with Python 3, also interpreted but ~30 times slower than Rust. I know that Python 3 can do some runtime stuff that Nodejs can't, but I wonder whether that's worth so much performance. Maybe if the answer is that you would include C mo…

Node uses V8, which does JIT compilation, while CPython is a straight bytecode interpreter. A better point of comparison would be PyPy

Are there any CPython vs PyPy benchmarks?

Re: Rust is now overall faster than C in benchmarks

#210

Post by jolux » Mon, Jan 04, 2021, 12:03 AM UTC

jandrewrogers wrote: post #61

jolux wrote: post #11

I think benchmarking C vs C++ vs Rust must only really be useful for researchers. They’re all making a similar tradeoff for performance: forcing you to consider how you use memory. Does anyone work in a field where the performance difference between these specific three platforms matters? I’m genuinely curious. Edit: also, if you could explain briefly why and what makes particular choices out of the three unsuitable,…

It matters for real-world software development, though the reason may not be intuitive. In theory, for any particular bit of software, you can write code in any of these three languages that has nearly identical performance. In practice, the complexity of expressing equivalent performance can vary considerably depending on what you are trying to do. There are finite limits to the complexity cost developers are willin…

This is a fantastic comment and I think it gives me a much better understanding of the tradeoffs here. It seems like which is the best choice depends a lot on what you're trying to express and how well the tool fits the problem you're trying to solve, and a lot less on what the absolute capabilities of a given language are.