Live data from Hacker News

Rust is now overall faster than C in benchmarks

benchmarksgame-team.pages.debian.net

401–410 of 445 posts

Re: Rust is now overall faster than C in benchmarks

#401

Earlier quoted context omitted.

The reason that C programs often perform well is that it’s so incredibly hard to do anything at all in C (especially something reliable) that one can usually only do the simplest thing possible and this typically means simple data structures, simple algorithms and arrays. In many ways, modern CPUs are particularly designed to run the machine code generated by C compilers on typical C code like this. Pointers and memo…

It's amazing that you could write this much and have it be 100% wrong.

From the guidelines[1]:

> Be kind. Don't be snarky.

> (…)

> When disagreeing, please reply to the argument instead of calling names.

> (…)

> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

You’re being downvoted because your comment doesn’t add to the discussion. You’re (rudely) telling the parent poster they are wrong while providing no basis for the assertion.

[1]: https://news.ycombinator.com/newsguidelines.html

Re: Rust is now overall faster than C in benchmarks

#402
post #400

Earlier quoted context omitted.

You can do better than that, but I suppose I don't expect more than an insubstantial pithy quip from you. Add analysis to each benchmark. Require submissions to come with analysis. Or more minimally, make the existing disclaimers on the web site more discoverable through one of any number of means, up to taste.

> Add analysis to each benchmark. Perhaps you'd like to take on that task? > Require submissions to come with analysis. Which raises the barrier for program contributors and presumably would require me to judge whether their analysis was acceptable? Me? Really? ;-) > disclaimers on the web site more discoverable The problem is that no one wants to "discover" disclaimers. We want to see something that supports whateve…

You asked me what could be done. I answered. I'm sure you have your reasons why you don't do more. But you asked.

> The problem is that no one wants to "discover" disclaimers.

Again repeating something I already said. It is indeed a problem. There are ways to address it. I personally think you do the absolute minimum.

Re: Rust is now overall faster than C in benchmarks

#403
post #80

Earlier quoted context omitted.

Nothing stops someone from copying and submitting other implementation's algorithm. There are multiple implementations of each benchmark for every language: • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It's possible that someone has already submitted both algorithms…

> Nothing stops someone Well, it looks like the submission process[0] and the maintainer[1] do, actually. [0]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov... [1]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov...

> "…but they don't accept Gmail addresses."

"Registration with @gmail.com addresses is blocked due to abuse, please register with another address. You can change it later, once signed up and verified."

However, apparently, you can register using a Google, Bitbucket or GitLab.com sign-in.

Re: Rust is now overall faster than C in benchmarks

#404
post #400

Earlier quoted context omitted.

> Add analysis to each benchmark. Perhaps you'd like to take on that task? > Require submissions to come with analysis. Which raises the barrier for program contributors and presumably would require me to judge whether their analysis was acceptable? Me? Really? ;-) > disclaimers on the web site more discoverable The problem is that no one wants to "discover" disclaimers. We want to see something that supports whateve…

You asked me what could be done. I answered. I'm sure you have your reasons why you don't do more. But you asked. > The problem is that no one wants to "discover" disclaimers. Again repeating something I already said. It is indeed a problem. There are ways to address it. I personally think you do the absolute minimum.

> It is indeed a problem. There are ways to address it.

Please make a specific suggestion.

Re: Rust is now overall faster than C in benchmarks

#405
post #362
post #353

Earlier quoted context omitted.

Not such a big deal in embedded, where the compiler and hardware is always the same, which means that the behavior is still predictable even if undefined

This is not true. Undefined behaviour can mean that ``` bool b; if (b) printf("1 "); if (!b) printf("2 "); ``` might print `1 2 `.

It's not like that is hard to fix, and modern compilers will flag uninitialized variables.

Use your brain and use your tools. C is not really that hard for a large number of problem sets.

Re: Rust is now overall faster than C in benchmarks

#406

Earlier quoted context omitted.

> I would’ve thought most commercial C programs would have heavily used linked lists, hashes (dictionary), and binary search trees all over the place. Every large scale C code base I have worked on has had these things, and they were used as you say all over the place. If I were to encounter a code base that didn't have these things I would wonder why it was written in C in the first place.

Conversely, I have only seen arrays being used, and never linked lists/hash maps in the few embedded code bases I have worked on.

And I have run across, and implemented, both linked lists, and arrays, in embedded systems. The implementation depended on what made the most sense for the particular problem being addressed.

Re: Rust is now overall faster than C in benchmarks

#407

Apart from those benchmark games a lot of real world C is a lot less performant than people think it might be. I spent a fair amount of time reviewing C code in the last 5 years - and things that pop up in nearly every review are costly string operations. Linear counts due to the use of null terminated strings and extra allocations for substrings to attach null terminators, or just deep copies because ownership can’t…

It is funny when I think of the hundreds of thousands of lines of code I have shipped in my long career (all embedded) and how little of it ever had to handle a string.

I agree, C is a really poor choice for string or other human readable content handling.

But if you have to read-shift-mask-write bytes to hardware control registers, it is a pretty easy language to use, and faster than assembly language in 90% of the cases, with modern compilers. That last note was less true in the mid 1980s, but by the mid-1990s, compiler optimization had gotten to be quite good.

Cellular modem code that runs on DSPs is written in C. Maybe someday that will be written in Rust. We'll see.

Re: Rust is now overall faster than C in benchmarks

#408
post #383

Earlier quoted context omitted.

The Haskell code on some pathological examples of these implementations that I have seen has so many unsafe construct, strictness annotations, inlining annotations and so forth that it's practically C in a different syntax. It is not idiomatic Haskell at all and loses all of the touted benefits. Is there a separate benchmark that only accepts idiomatic code?

Is there a GHC compiler switch that ensures only "idiomatic code" is compiled? One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of “reasonably idiomatic” Haskell. However, we have made abundant use of strictness annotations, explic…

It shows the performance of that variant of Haskell.

But when using that variant all the correctness guarantees that Haskell is known for are no longer on the table.

Re: Rust is now overall faster than C in benchmarks

#409
post #404

Earlier quoted context omitted.

You asked me what could be done. I answered. I'm sure you have your reasons why you don't do more. But you asked. > The problem is that no one wants to "discover" disclaimers. Again repeating something I already said. It is indeed a problem. There are ways to address it. I personally think you do the absolute minimum.

> It is indeed a problem. There are ways to address it. Please make a specific suggestion.

I did. You just didn't like their costs. Which is reasonable. They do have costs. I just happen to think they are worth it.

For analysis, it increases review time and increases the burden on contributors. The extent to which any one person can review all analyses isn't clear to me, and they would likely need to rely on contributors to self regulate them. But I would expect the maintainers to review them for some minimal level of quality. Being more specific than this would require a more thorough proposal. Given my time constraints and how much I dislike interacting with you, it's not something I can do right now.

As for increasing contributor barriers, yes, adding an analysis requirement would do that. However, I think you could compensate for that partially by reducing existing barriers. I've submitted code to your game before, and I'm unlikely to ever do it again. One reason is that interacting with you was unpleasant for me. But I found the interface difficult to navigate (that has since changed) and the rules difficult to discover. I think barriers could be reduced by switching to github (one of the reasons I use github) with more automated checks on submissions and better documented rules. (At the time when I worked on a submission it was unclear for example how to submit a Rust program that used external crates.)

Invariably though, providing an analysis is still more work. IMO, I think it would add rnough value to be worth it. I imagine many amalyses would build on top of others, similar to how code submissions do that today. As can be seen in any thread about the Benchmark Game, there are tons of comments that misunderstand things or make wild guesses. (Followed up invariably a day later with coy quips from you.) Some of those comments are from people that are unlikely to be helped by my analysis idea. But a lot aren't. I think a lot of people would benefit from an explanation of the performance characteristics of the program.

I generally try not to publish benchmarks without analysis. Here's one example.[1] I care a lot about it and I think it has immense benefits in terms of educating others. But reasonable people can disagree. You might have different values than me, which influences whether my suggestions are applicable to you or not.

As for the disclaimer, I don't really see the point of being more specific. There isn't much to this one? A simple suggestion might be to put it at the top of each page or even in the description of each benchmark. I don't know how much this would help things. Maybe people would still just ignore it.

People regularly misinterpret benchmarks and especially those in your Benchmark Game. From what I can tell, you as the maintainer have done very little to address that. Maybe you don't care. Maybe you think it isn't your problem to solve. Maybe you think it can't be solved or even improved, and therefore it isn't worth doing anything. I don't know. We might just disagree, which is fine.

Invariably, you can just respond and tell me to go do it. I think that's a cheap response, but I know you're fond of it. I've been thinking a lot about it and considering starting my own benchmark. But it will likely be a few years before I'll get to it. It is a trying daunting amount of work.

[1] - https://blog.burntsushi.net/ripgrep/

Re: Rust is now overall faster than C in benchmarks

#410

Earlier quoted context omitted.

Looked at the git repo https://salsa.debian.org/benchmarksgame-team/benchmarksgame and it seems there's only one committer. So if all scripts are written by the same guy, then his biases are all over the place, and it needs more code diversity before we can arrive to any conclusion. Am I missing something?, is there a list somewhere of the people who have contributed to this?

Click on any of the benchmark programs and it will have the contributors' names at the beginning.

Thanks!
Post reply on HN