Cross-platform Rust rewrite of the GNU coreutils • Hacker News Board

Re: Cross-platform Rust rewrite of the GNU coreutils

#231

Post by pcwalton » Tue, Mar 22, 2016, 7:22 PM UTC

weberc2 wrote: post #220

pcwalton wrote: post #95

Earlier quoted context omitted.

Disappointing to see Hipp make that argument. It's trivially refuted. Yes, all programming languages allow the programmer to write bugs. But languages very much vary in how many , and what kinds of bugs programmers write in practice . Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man. If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in…

> If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in the important areas in practice than Rust (plus kcov [1] if you like) does. I don't have a dog in this fight, but I don't see how the burden of proof is on Hipp rather than the folks proposing the change. In other words, shouldn't the "rewrite it in Rust" folks have to prove that the cost of their propos…

> In other words, shouldn't the "rewrite it in Rust" folks have to prove that the cost of their proposed rewrite will be justified?

Agreed, they do. But that it hasn't been proven to make things better doesn't mean it will make things worse. It just means that we don't know enough to say. The right way to answer the "would this software have fewer bugs if it were rewritten in Rust?" question requires a detailed look at what bugs the software has empirically encountered.

Re: Cross-platform Rust rewrite of the GNU coreutils

#232

Post by all2well » Tue, Mar 22, 2016, 7:23 PM UTC

nickpsecurity wrote: post #175

steveklabnik wrote: post #118

Earlier quoted context omitted.

The Rust standard library does assume that allocation succeeds. The language itself knows nothing about the heap, and so you can write allocators that do whatever you wish. Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure. I am less knowledgeable about OSX and Windows. I cannot speak to SQlite in this rega…

Holy crap, I'd never have imagined that... A design that assumes allocation always succeeds flies in the face of decades of safety/security-critical coding wisdom. I strongly recommend the team revisit and change that somehow to account for failures, NULL's, whatever. Actually, same anywhere a failure-prone, esp hardware, resource is acquired. C apps can handle this issue so Rust should as well if it's to replace the…

At least for the nightly builds, you can specify a closure to run when OOM happens. The only restriction is that it cannot return, but you can "recover" from panics in the nightly standard library as well. The thinking was, I think, that in most cases the default behavior of abort is the correct behavior. Recovering from a failed malloc is really only relevant in large allocations, and there are multiple paths (including directly using __rust_allocate) to recovery in that case.

Furthermore, keep in mind that the _pure language_ Rust has no concept of dynamic allocation (apart from "language elements" like __rust_allocate, which are kind of like a pre-linker).

Re: Cross-platform Rust rewrite of the GNU coreutils

#233

Post by dikaiosune » Tue, Mar 22, 2016, 7:23 PM UTC

aidenn0 wrote: post #152

yberreby wrote: post #124

Earlier quoted context omitted.

Autocompletion: YouCompleteMe + racer + racerd Source formatting: rust.vim + rustfmt Automatic building: https://github.com/passcod/cargo-watch Error display: tmux, terminator or iTerm2 split planes Rust has pretty good tool support in vim and Atom, for example (can't speak for the rest). We don't have stellar IDE support yet, but it's on the to-do list: https://www.rust-lang.org/ides.html Personally, I don't miss ID…

> Error display: tmux, terminator or iTerm2 split planes Split panes are a poor substitute for actual error display integration; highlighting the errors in the source-code viewer, combined with a cross-linked list of errors is far more useful.

The linter-rust plugin for Atom provides this, as does the RustyCode plugin for VSCode.

Re: Cross-platform Rust rewrite of the GNU coreutils

#234

Post by frutiger » Tue, Mar 22, 2016, 7:24 PM UTC

jordigh wrote: post #214

mapleoin wrote: post #22

What are the licensing implications for this kind of work? I assume the authors used GNU coreutils as more than just inspiration. They probably read all the original code and reused some of the solutions (obviously ported to Rust). Shouldn't the derivative work still be covered by the GPL?

Reading GPL'ed code and reimplementing it makes a good case for why the rewrite should also be GPL'ed. I don't know if they did this here, but if so, they should GPL their version as well. For GNU Octave, we stress very strongly that anyone who has read Matlab's source code is ineligible to contribute to Octave. This is because, should it ever come down to it, we want to be able to ascertain that our implementation i…

> This is because, should it ever come down to it, we want to be able to ascertain that our implementation is completely original

I appreciate that this is how it works today, but isn't that a completely outrageous idea? A well read, well travelled person will have seen countless things that will influence their future behaviours. It is not uncommon to completely forget a particular source of inspiration (sometimes we falsely attribute to someone else, and even other times we attribute it to ourselves!)

Re: Cross-platform Rust rewrite of the GNU coreutils

#235

Post by frutiger » Tue, Mar 22, 2016, 7:27 PM UTC

dikaiosune wrote: post #159

kazinator wrote: post #104

Complete rubbish: Many GNU, Linux and other utils are pretty awesome, and obviously some effort has been spent in the past to port them to Windows. However, those projects are either old, abandoned, hosted on CVS, written in platform-specific C, etc. Rust provides a good, platform-agnostic way of writing systems utils that are easy to compile anywhere, and this is as good a way as any to try and learn it. Cygwin has…

Is the cygwin port just some different build files, or is it genuinely different code? A Rust codebase probably wouldn't (in most cases) need to be modified to run on different supported platforms.

I'm not going to comment about cygwin (because its build process is not as transparent) but instead about msys2, the spiritual successor to cygwin. See for yourself: https://github.com/Alexpux/MSYS2-packages/blob/master/coreut... is the `pacman` style `PKGBUILD` used to build the `coreutils` package.

Re: Cross-platform Rust rewrite of the GNU coreutils

#236

Post by dalke » Tue, Mar 22, 2016, 7:27 PM UTC

ktRolster wrote: post #155

kbenson wrote: post #76

Earlier quoted context omitted.

Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate whatever machine code it wants without it being called a compiler bug. That’s an important finding. But as it happens, no modern compilers that we know of actually interpret any of the SQLite source code in…

At some point some popular compiler is going to make a subtle but important change to some undefined behavior that's not going to be immediately obvious as to it's repercussions, and the fallout will be massive. Wait until you realize that the length of a byte in C is not clearly defined. Someday a processor will come along where a byte is 6 bits, and the fallout will be massive. (Really, it's happened before). No, a…

> The reality is unless you are using a formally-defined language like ML, you are relying on undefined behavior in your language.

The CompCert C verified compiler is a verified compiler which supports "almost all of the ISO C90 / ANSI C language" - http://compcert.inria.fr/ .

It isn't C. It is a formally-defined language that is very similar to C. It is otherwise nothing like ML.

Re: Cross-platform Rust rewrite of the GNU coreutils

#237

Post by pcwalton » Tue, Mar 22, 2016, 7:29 PM UTC

adrusi wrote: post #148

cbsmith wrote: post #131

Earlier quoted context omitted.

Really, if your project intends to be portable C code, you need to compile it with undefined behaviour generating an error.

Undefined behavior is not always identifiable through static analysis. Obviously it can be checked against at runtime, but that's actually quite expensive. It would, for example, include bounds checks for everything, and overflow checks on all signed arithmetic.

That's not the worst of it: the truly intractable part is preventing use-after-free UB. The only ways to do this are (a) remove malloc from your language; (b) add a lifetime system (incompatible with all existing C libraries); (c) add a garbage collector (which most projects written in C will not accept for performance reasons).

Re: Cross-platform Rust rewrite of the GNU coreutils

#238

Post by Animats » Tue, Mar 22, 2016, 7:29 PM UTC

Yes! No more buffer overflow exploits in the core utils.

Re: Cross-platform Rust rewrite of the GNU coreutils

#239

Post by steveklabnik » Tue, Mar 22, 2016, 7:33 PM UTC

spaceheeder wrote: post #207

steveklabnik wrote: post #184

Earlier quoted context omitted.

Redox uses a different, BSD style coreutils, not GNU style.

Please pardon my ignorance, but what are some of the key differences?

I am not an expert, but the general difference, in my understanding, is that the BSD coreutils is significantly "lighter", both in features and in code size.

Re: Cross-platform Rust rewrite of the GNU coreutils

#240

Post by pcwalton » Tue, Mar 22, 2016, 7:34 PM UTC

Alupis wrote: post #113

thegeomaster wrote: post #12

Earlier quoted context omitted.

Not sure why you're being downvoted. I don't see why we shouldn't be moving to languages like Rust given the chance, as C makes it far more difficult to write safe and correct code.

> I don't see why we shouldn't be moving to languages like Rust given the chance, as C makes it far more difficult to write safe and correct code I agree in principal, however the timing is wrong. Rust is a very new language (less than 6 years old), and is undeniably totally unproven. It is the latest "buzz" language, and may not be around long term... nobody knows. What-more, a full-fledged re-write of CoreUtils in…

> Flatly, re-writing a several decade's old, matured codebase in today's flavor-of-the-week language is not a good idea.

It's odd that so many in the computing industry are unwilling to move on from a language from 1978. We think of ourselves as one of the most fast-moving industries, but we have this odd reverence for early C and Unix that makes us stubborn and resistant to change. The fact is: we didn't know how to do some things properly in 1978. We know more about programming language design now. (Even Rob Pike would presumably agree—that's why he created Go!)

To be sure, we shouldn't just rewrite things in new languages for no reason. I think many segments of our industry are too fad driven. But to me the right thing is simple: Let's evaluate new technologies on their merits. Rust may well be worse than C! But if it is, let's figure out why, and say so explicitly.