Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Rust for Low-Level Linux Programming?

groveronline.com

Post reply on HN
Print view View original

201–210 of 231 posts

Re: Why Rust for Low-Level Linux Programming?

#201

Post by igouy »

burntsushi wrote: post #157
bjourne wrote: post #131

Earlier quoted context omitted.

Rust is also slower in binarytrees, regexdna and fasta. SSE is not one "barely used corner case" because huge amounts of performance critical code takes advantage of it. Edit: To explain why I don't believe you when you say that "Post compilation they are functionally identical [in performance]" is because if it were so, you would just transliterate the C solutions to the Rust equivalents and it would run as fast as…

Did you know Rust was quite a bit faster than C in regexdna merely a few months ago? It didn't get slower because of Rust. The algorithms employed are radically different. My hope is that the regex library has already regained performance, but until the benchmark game is updated (which is on us, not the benchmark game maintainer), I suppose we'll have to suffer the pedants! Or perhaps, you might look at single thread…

> It didn't get slower because of Rust.

Do you mean the program became relatively slower because of changes you've made to the regex crate?

Wasn't the program relatively faster because you wrote the regex crate to use Aho-Corasick for the matches required by the regex-dna task?

Re: Why Rust for Low-Level Linux Programming?

#202

Post by igouy »

nemothekid wrote: post #128
bjourne wrote: post #122

Earlier quoted context omitted.

I wrote "what can you do" because you are supposed to use some modicum of common sense when reading the numbers on The Benchmark Game. E.g in one of the benchmarks PHP beats both C and Rust, so you need to apply common sense to understand that that result is an outlier. I didn't cherry-pick; in 5/10 benchmarks, C is twice as fast as Rust. > rust code shouldn't have any more overhead. But it appears that it have. > We…

> I didn't cherry-pick; in 5/10 benchmarks, C is twice as fast as Rust. Making the claim that C is twice as fast as Rust because of 5/10 benchmarks in the "benchmark game" shows an incredible lack of common sense to me. In 5/10 benchmarks, the benchmark games claims Go has equal if not better performance than Rust. Am I supposed to believe now, that a managed, garbage collected, 6-year-old compiler, language is as fa…

>> Don't back up your claim with flawed benchmarks.

:-)

" How fast is Rust? Fast! Rust is already competitive with idiomatic C and C++ in a number of benchmarks (like the Benchmarks Game and others)."

https://www.rust-lang.org/faq.html#performance

Re: Why Rust for Low-Level Linux Programming?

#203

Post by igouy »

Manishearth wrote: post #175
bjourne wrote: post #122

Earlier quoted context omitted.

I wrote "what can you do" because you are supposed to use some modicum of common sense when reading the numbers on The Benchmark Game. E.g in one of the benchmarks PHP beats both C and Rust, so you need to apply common sense to understand that that result is an outlier. I didn't cherry-pick; in 5/10 benchmarks, C is twice as fast as Rust. > rust code shouldn't have any more overhead. But it appears that it have. > We…

> some modicum of common sense when reading the numbers on The Benchmark Game yes, this involves checking what the benchmarks are actually measuring. In this case, it is how much faster SIMD makes things. Factor that in, or rewrite the programs with SIMD in rust, and it should come out to be the same. > But it appears that it have. Have you not been listening? It doesn't. The speed differences you quote are due to si…

> Have you not been listening? It doesn't. The speed differences you quote are due to simd.

That should be easy to demonstrate!

Please quote the lines in the source-code of these fannkuch-redux and reverse-complement programs that show SIMD use --

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Re: Why Rust for Low-Level Linux Programming?

#204

Post by Manishearth »

notalaser wrote: post #189
Manishearth wrote: post #178

Earlier quoted context omitted.

> static analyzers Rust doesn't exactly need these, no? Most static analysis in C/++ is safety/UB focused. Rust doesn't need this, unless you're going to spend a lot of time with `unsafe` code. Rust does have clippy, a lint library with >150 lints which catch things ranging from correctness to style to safety issues. I'm one of the maintainers, so I'm biased, but I've personally found it to be much better than its eq…

Static analyzers are good for a lot more than just finding potentially unsafe memory access. In fact, memory access bugs are typically just the low-hanging fruits that static analyzers find (and which, most of the time, you can find by code review, assuming your team consists of more than two developers and that they actually get some sleep every once in a while). It's issues related to timing constraints, incomplete…

I know they are, I'm saying that the main attraction is memory access stuff (at least, for me it was when I used to use them).

For the rest, Rust does have static analysis tooling of the kind you describe in the form of clippy. There's still a lot that can be done, but it's already quite helpful and catches all kinds of things.

Re: Why Rust for Low-Level Linux Programming?

#205

Post by burntsushi »

igouy wrote: post #201
burntsushi wrote: post #157

Earlier quoted context omitted.

Did you know Rust was quite a bit faster than C in regexdna merely a few months ago? It didn't get slower because of Rust. The algorithms employed are radically different. My hope is that the regex library has already regained performance, but until the benchmark game is updated (which is on us, not the benchmark game maintainer), I suppose we'll have to suffer the pedants! Or perhaps, you might look at single thread…

> It didn't get slower because of Rust. Do you mean the program became relatively slower because of changes you've made to the regex crate? Wasn't the program relatively faster because you wrote the regex crate to use Aho-Corasick for the matches required by the regex-dna task?

> Do you mean the program became relatively slower because of changes you've made to the regex crate?

Yes. The underlying reasoning is complex. When the regex crate got a lazy DFA (similar to the one used by RE2), the vast majority of regexes got significantly faster. Some got slower. This one in particular from regex-dna:

    >[^\n]*\n|\n
Before the lazy DFA, compile time analysis would notice that all matches either start with `>` or `\n` and do a fast prefix scan for them. Each match of `>` or `\n` represents a candidate for a match. Candidates were then verified using something similar to the Thompson NFA, which is generally pretty slow, but the prefix scanning reduced the amount of work required considerably.

Once the lazy DFA was added, the prefix scanning was still used, but the lazy DFA was used to verify candidates. It's faster in general by a lot, but, the lazy DFA requires two scans of the candidate: one to find the end position and another to find the start position. That extra scan made processing this regex (on the regex-dna input) slightly slower.

I've since fixed some of this by reducing a lot of the match overhead of the lazy DFA, so my hope is that it's back to par, but I haven't done any rigorous benchmarking to verify that.

> Wasn't the program relatively faster because you wrote the regex crate to use Aho-Corasick for the matches required by the regex-dna task?

Aho-Corasick is principally useful for the second phase of regex-dna, e.g., the regexes that look like `ag[act]gtaaa|tttac[agt]ct`. (In the last phase, all the regexes are just single byte literals, so neither Aho-Corasick nor the regex engine should ever be used.) Performance here should stay the same.

On that note, I have a new nightly-only algorithm called Teddy that uses SIMD[1] (which replaces the use of Aho-Corasick for those regexes) and is a bit faster. I got the algorithm from the Hyperscan[2] project, which also does extensive literal analysis to speed up regexes.

To clarify, this optimization is generally useful because a lot of regexes in the wild have prefix literals. Even something like `(?i:foo)\s+bar` can benefit from it, since `(?i:foo)` expands to FOO, FOo, FoO, Foo, fOO, fOo, foO, foo, which can of course be used with Aho-Corasick (and also my new SIMD algorithm).

One also must wonder how well a C program using PCRE2's JIT would fair on the benchmarks game. From my experience, it would probably be near the top. It's quite fast!

[1] - https://github.com/rust-lang-nursery/regex/blob/master/src/s...

[2] - https://github.com/01org/hyperscan

Re: Why Rust for Low-Level Linux Programming?

#206

Post by DonaldFisk »

RealityVoid wrote: post #200
DonaldFisk wrote: post #198

Earlier quoted context omitted.

I don't think C (or C++) should be used for autonomous vehicles at all, as it is known to be unsafe, but if it is, the MISRA C guidelines or something similar should be used to help prevent certain kinds of bugs. Almost any other statically typed language, along with similarly strict guidelines, would be preferable to C, but there is no ideal language. Rust still allows dynamic heap memory allocation and recursive fu…

MISRA is already used extensively in the auto industry. But i guess what I was trying to say is that while it helps, it can easily be tricked while a compiler designed with the safety measures MISRA promotes already baked into it will not let you do certain things.

Agreed.

Re: Why Rust for Low-Level Linux Programming?

#207

Post by bjourne »

Manishearth wrote: post #199
bjourne wrote: post #192

Earlier quoted context omitted.

No. In the first table Rust has 1874 and C++/clang 1722. The latter number is lower. C++ with clang beats Rust. In the second table all C and C++ versions beats Rust. 618, 749, 755 and 735 vs 877. That is a very big difference. You can also run the fucking benchmarks yourself and see for yourself. I have linked to lots of benchmarks showing C spanking Rust. None has shown any fair benchmarks were Rust is as fast as C…

That's a ... very small difference. And again, probably due to implementation differences. I'm not claiming C doesn't beat Rust, I'm just saying by very little -- Rust is practically just as fast, within the margin of error that microbenchmarks have. You have been belting out claims that Rust is 2x slower -- clearly false. Rust may be 5% slower -- which ... doesn't really matter. Look at wycats' talk on fast_blank. T…

> That's a ... very small difference. And again, probably due to implementation differences.

735 / 618 = 1.19 So Rust is at least 19% slower than C even without involving SIMD intrinsics. You wrote "your rust code shouldn't have any more overhead" but in all the benchmarks it has!

> You have been belting out claims that Rust is 2x slower -- clearly false.

Clearly not, since it is on all the SIMD-using benchmarks.

> Look at wycats' talk on fast_blank. That's a real world example that's faster than C. Rust used to be faster than c on the regex benchmark at one point, as burntsushi pointed out.

Because it is comparing different regex engines, not language performance. I said you should apply "common sense" to The Benchmark Game's numbers.

Here's the thing, you guys can easily prove me wrong. Prove that Rust has zero cost abstractions by taking any small C benchmark, transliterate it to Rust code and profile it. If it is as fast, I'm proven wrong. If it is slower you are proven wrong.

Re: Why Rust for Low-Level Linux Programming?

#208

Post by igouy »

burntsushi wrote: post #205
igouy wrote: post #201

Earlier quoted context omitted.

> It didn't get slower because of Rust. Do you mean the program became relatively slower because of changes you've made to the regex crate? Wasn't the program relatively faster because you wrote the regex crate to use Aho-Corasick for the matches required by the regex-dna task?

> Do you mean the program became relatively slower because of changes you've made to the regex crate? Yes. The underlying reasoning is complex. When the regex crate got a lazy DFA (similar to the one used by RE2), the vast majority of regexes got significantly faster. Some got slower. This one in particular from regex-dna: >[^\n]*\n|\n Before the lazy DFA, compile time analysis would notice that all matches either st…

> One also must wonder how well a C program using PCRE2's JIT would fair on the benchmarks game.

Let's hope some C and C++ programmers take up the challenge ;-)

Re: Why Rust for Low-Level Linux Programming?

#209

Post by Manishearth »

bjourne wrote: post #207
Manishearth wrote: post #199

Earlier quoted context omitted.

That's a ... very small difference. And again, probably due to implementation differences. I'm not claiming C doesn't beat Rust, I'm just saying by very little -- Rust is practically just as fast, within the margin of error that microbenchmarks have. You have been belting out claims that Rust is 2x slower -- clearly false. Rust may be 5% slower -- which ... doesn't really matter. Look at wycats' talk on fast_blank. T…

> That's a ... very small difference. And again, probably due to implementation differences. 735 / 618 = 1.19 So Rust is at least 19% slower than C even without involving SIMD intrinsics. You wrote "your rust code shouldn't have any more overhead" but in all the benchmarks it has! > You have been belting out claims that Rust is 2x slower -- clearly false. Clearly not, since it is on all the SIMD-using benchmarks. > L…

Rust can use simd too. It doesn't in those benchmarks. Please apply the common sense you keep harping about. Claiming rust is 2x slower because of that benchmark is a falsehood.

Re:regex: my point exactly. Most microbenchmarks are prone to slight differences in the implementation causing issues (and you can rarely translate code exactly, especially to something like rust which often requires a different structure of code from C. Same for any two other langauges). 19% is well within this error box.

The fast_blank thing is this example. fast_blank is a carefully hand optimized C extension whose main purpose is being super fast. A mostly naive Rust one-liner beat it (not by much iirc, perhaps 10%, but thats within the error box im talking about). It didn't use parallelism or anything fancy. They weren't even trying to beat C. I provided this proof already.

I could try fixing that benchmark you linked to -- the rust version looks like it could be optimized further. Not sure if its worth it, really. I don't put much stock in microbenchmarks for anything other than order of magnitude comparisons.

Re: Why Rust for Low-Level Linux Programming?

#210

Post by bjourne »

Manishearth wrote: post #209
bjourne wrote: post #207

Earlier quoted context omitted.

> That's a ... very small difference. And again, probably due to implementation differences. 735 / 618 = 1.19 So Rust is at least 19% slower than C even without involving SIMD intrinsics. You wrote "your rust code shouldn't have any more overhead" but in all the benchmarks it has! > You have been belting out claims that Rust is 2x slower -- clearly false. Clearly not, since it is on all the SIMD-using benchmarks. > L…

Rust can use simd too. It doesn't in those benchmarks. Please apply the common sense you keep harping about. Claiming rust is 2x slower because of that benchmark is a falsehood. Re:regex: my point exactly . Most microbenchmarks are prone to slight differences in the implementation causing issues (and you can rarely translate code exactly , especially to something like rust which often requires a different structure o…

> Rust can use simd too.

No it can't. Either accept that the nightly build of Rust is not the Rust we are talking about or stop discussing with me.

> Re:regex: my point exactly.

The problem with regex libraries are that they are to big so therefore doesn't reveal so much about inherent language performance.

> Most microbenchmarks are prone to slight differences in the implementation causing issues (and you can rarely translate code exactly, especially to something like rust which often requires a different structure of code from C. Same for any two other langauges).

Yes, obviously the implementation defines performance. That's what I wrote in the other thread part: "this discussion is about Rust vs C. Or rather clang 3.6.2/gcc 5.2.1 vs Rust 1.9.0 since language performance is very implementation dependent"

And fwiw, you can easily transliterate C code to C++ or to asm.

> 19% is well within this error box.

What error box? 19% is a huge difference.

> The fast_blank thing is this example. fast_blank is a carefully hand optimized C extension whose main purpose is being super fast.

I don't know what fast_blank is. Is it this https://github.com/SamSaffron/fast_blank/blob/master/ext/fas... C code wycatz managed to rewrite faster in Rust? That C code isn't well-optimized at all...

> I don't put much stock in microbenchmarks for anything other than order of magnitude comparisons.

Does that mean it is impossible to prove to you that C is at least 2x faster than Rust since twice is less than one order of magnitude?

Post reply on HN