Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cross-platform Rust rewrite of the GNU coreutils

github.com

Post reply on HN
Print view View original

361–370 of 498 posts

Re: Cross-platform Rust rewrite of the GNU coreutils

#361

Post by reynoldsbd »

pcwalton wrote: post #116
jfoutz wrote: post #103

Earlier quoted context omitted.

The quality of tooling, and the ability of experts to verify the output of machine code is a really important point. I think I'd agree, rust at this point would hold back an elite developer like Richard Hipp. The promise of rust, which may or may not be realized is pushing some very common problems down to the compiler. All code has bugs, so the compiler probably does things wrong in some cases. As the tools mature,…

I think the idea that "elite" developers can write bug-free C (or even just network-facing C free of security-sensitive memory safety problems) is pretty well refuted at this point. Now you can write bug-free C if you're willing to spend enormous time and money on testing: this is to a first approximation what SQLite did. But that only makes economic sense for a small minority of projects. Just putting "elite" develo…

It will be interesting to see if the formal proof/verification work that is being done by NICTA for the seL4 project will mature into something that can be used practically elsewhere in industry.

https://sel4.systems/

Re: Cross-platform Rust rewrite of the GNU coreutils

#362

Post by coliveira »

andrewflnr wrote: post #264
weberc2 wrote: post #220

Earlier quoted context omitted.

> If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in the important areas in practice than Rust (plus kcov [1] if you like) does. I don't have a dog in this fight, but I don't see how the burden of proof is on Hipp rather than the folks proposing the change. In other words, shouldn't the "rewrite it in Rust" folks have to prove that the cost of their propos…

Sure, the primary burden of proof is on those proposing a change. However, any time you stand up and make an argument, the burden is on you to make sure it actually makes sense, and that goes for both sides.

Anything that is stated without proof can be refuted without proof. Why you think your opinion should be regarded at a higher standard than anybody else's?

Re: Cross-platform Rust rewrite of the GNU coreutils

#363

Post by vardump »

andrepd wrote: post #193
m0th87 wrote: post #168

Earlier quoted context omitted.

> Difficulty is relative. If you don't study modern C idioms your C code will be crap. Same goes for Rust. That's not a valid argument for why better tooling can't help alleviate some of the difficulty. > Honestly using higher-level languages is the same mentality as taking a pill to magically lose weight. It's quick, but detrimental (to programmers ability) in long term. If that were true, the most effective program…

>If that were true, the most effective programmers would only code on assembly. No, because C is as fast as hand-coded assembly in most of the cases. The same can't be said of any high level language in comparison to C (save for C++ and Fortran).

> No, because C is as fast as hand-coded assembly in most of the cases.

Most hand-coded assembly is pretty slow. C isn't any pinnacle of performance either.

C programs are compiled to some non-existent abstract machine that ignores the real variations in memory architecture and CPU implementations. The compiled binary can't adapt at runtime.

I don't know of any C implementations that take advantage of runtime information for optimization purposes. So you end up with generated code with a lot of redundant computation, tests, branches and pointer dereferences (such as function pointer dereference that always refers to same address), just because they might be necessary with some input -- input that wasn't the case this time.

A single mispredicted branch is expensive. Say branch mispredict takes 15 cycles. That's enough to do up to 480 (32*15) 32-bit floating point operations on a single core. Ignoring runtime information takes us pretty far from anything you can call optimal code.

The current crop of compilers are also pretty bad at vectorizing anything complicated. Those cases it can be pretty trivial to beat the compiler by 2-10x, in some cases even 40x+ if your vectorization can also eliminate a lot of unpredictable branches.

Re: Cross-platform Rust rewrite of the GNU coreutils

#364

Post by caf »

pcwalton wrote: post #138
catawbasam wrote: post #128

Earlier quoted context omitted.

'Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man.' This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier or naive about software quality is something I find profoundly mis-guided.

> This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. "Rust doesn't eliminate all bugs" is a rephrased version of "Some well-formed rust programs will generate machine code that behaves differently from what the programmer expected." > That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier…

1. The amount of testing that SQLite undergoes is not economically feasible for most software.

But the argument isn't about "most software" - it's about SQLite in specific. And that's the crux of the issue. Saying that he believes it would be counterproductive to rewrite SQLite in Rust at this time is not saying that that would be true of all or most or some other programs.

Re: Cross-platform Rust rewrite of the GNU coreutils

#365

Post by biot »

mc808 wrote: post #349
randommodnar wrote: post #259

MIT license? Is open-source dying?

I was going to say it only takes about 30 seconds to fork and change the license if you want to. Then I decided to check if someone had already done so. Yep. https://github.com/ivegotasthma/coreutils/commit/4c7dcbd912a... I didn't count, but that may have taken less than 30 seconds.

Is that valid? The original license had:

  > The above copyright notice and this permission notice shall be
  > included in all copies or substantial portions of the Software.
As those have now been stripped away, it seems like a license violation.

Re: Cross-platform Rust rewrite of the GNU coreutils

#366

Post by jstimpfle »

comex wrote: post #350
jstimpfle wrote: post #324

Earlier quoted context omitted.

Help me out here, I don't know much about Rust. Which bad programs does Rust rule out? I know about the borrow checker, but I don't think ownership bugs is a type of bug that Mr. Hipp frequently produces. It's a program design issue -- not something you think about at every single line you write. A well-designed program does not do many ownership transfers. As someone else noted, out-of-bounds errors are sadly a pain…

> I know about the borrow checker, but I don't think ownership bugs is a type of bug that Mr. Hipp frequently produces. The borrow checker eliminates use-after-free. And use-after-free is one of the most common types of vulnerability exploited in practice today, if not the single most common. (For evidence, look at reports about Pwn2Own.) Index checking is quite cheap for most programs, and LLVM is good at eliminatin…

I don't think you can count use-after-free as exploitable. Sure, if you have them (and I think those cases can largely be ruled out by good design), it leads to crashes. But for memory type exploits you need control over the value in it. Not a security specialist but I'm not aware of common attacks besides overflowing buffers.

In hot paths (like codecs, compression algorithms...) I'm sure you never want bounds checking. It can be optimized away for sequential loops of course, but not so easily in the case of data lookups.

Re: Cross-platform Rust rewrite of the GNU coreutils

#367

Post by steveklabnik »

jstimpfle wrote: post #366
comex wrote: post #350

Earlier quoted context omitted.

> I know about the borrow checker, but I don't think ownership bugs is a type of bug that Mr. Hipp frequently produces. The borrow checker eliminates use-after-free. And use-after-free is one of the most common types of vulnerability exploited in practice today, if not the single most common. (For evidence, look at reports about Pwn2Own.) Index checking is quite cheap for most programs, and LLVM is good at eliminatin…

I don't think you can count use-after-free as exploitable. Sure, if you have them (and I think those cases can largely be ruled out by good design), it leads to crashes. But for memory type exploits you need control over the value in it. Not a security specialist but I'm not aware of common attacks besides overflowing buffers. In hot paths (like codecs, compression algorithms...) I'm sure you never want bounds checki…

https://www.owasp.org/index.php/Using_freed_memory

Re: Cross-platform Rust rewrite of the GNU coreutils

#368

Post by cbsmith »

adrusi wrote: post #148
cbsmith wrote: post #131

Earlier quoted context omitted.

Really, if your project intends to be portable C code, you need to compile it with undefined behaviour generating an error.

Undefined behavior is not always identifiable through static analysis. Obviously it can be checked against at runtime, but that's actually quite expensive. It would, for example, include bounds checks for everything, and overflow checks on all signed arithmetic.

Trapping on an overflow register signal is very, very cheap, and there are standard libraries for it.

Some of this stuff can be caught at compile time. Like code that does signed addition of positive values and checks if they might be less.

Re: Cross-platform Rust rewrite of the GNU coreutils

#369

Post by steveklabnik »

reynoldsbd wrote: post #361
pcwalton wrote: post #116

Earlier quoted context omitted.

I think the idea that "elite" developers can write bug-free C (or even just network-facing C free of security-sensitive memory safety problems) is pretty well refuted at this point. Now you can write bug-free C if you're willing to spend enormous time and money on testing: this is to a first approximation what SQLite did. But that only makes economic sense for a small minority of projects. Just putting "elite" develo…

It will be interesting to see if the formal proof/verification work that is being done by NICTA for the seL4 project will mature into something that can be used practically elsewhere in industry. https://sel4.systems/

You might be interested in https://robigalia.org/

Re: Cross-platform Rust rewrite of the GNU coreutils

#370

Post by astrange »

pcwalton wrote: post #237
adrusi wrote: post #148

Earlier quoted context omitted.

Undefined behavior is not always identifiable through static analysis. Obviously it can be checked against at runtime, but that's actually quite expensive. It would, for example, include bounds checks for everything, and overflow checks on all signed arithmetic.

That's not the worst of it: the truly intractable part is preventing use-after-free UB. The only ways to do this are (a) remove malloc from your language; (b) add a lifetime system (incompatible with all existing C libraries); (c) add a garbage collector (which most projects written in C will not accept for performance reasons).

And once you've eliminated that, you have DoS bugs like forcing infinite loops, abandoned (referenced but unused) memory leaks, and worst-case hash table insertions. All of those are serious attacks for anything with a resource budget.
Post reply on HN