Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cross-platform Rust rewrite of the GNU coreutils

github.com

Post reply on HN
Print view View original

261–270 of 498 posts

Re: Cross-platform Rust rewrite of the GNU coreutils

#261

Post by pcwalton »

Alupis wrote: post #258
pcwalton wrote: post #240

Earlier quoted context omitted.

> Flatly, re-writing a several decade's old, matured codebase in today's flavor-of-the-week language is not a good idea. It's odd that so many in the computing industry are unwilling to move on from a language from 1978 . We think of ourselves as one of the most fast-moving industries, but we have this odd reverence for early C and Unix that makes us stubborn and resistant to change. The fact is: we didn't know how t…

> We think of ourselves as one of the most fast-moving industries We're fast-moving because our foundation is solid and not changing (ie. CoreUtils and gang). It's an assumption that these things "just work" with zero fuss and weirdness between systems. We build on-top of these systems, so changing them out from underneath us all is a dramatic shift. Perhaps Rust is the key to making these things better. I never clai…

> We're fast-moving because our foundation is solid and not changing (ie. CoreUtils and gang).

That isn't a definition of "fast moving" that I would use. It sounds like slow moving.

Why is innovation in the core layers of the system less legitimate than innovation in social media apps? Boeing has no problem upgrading their engines every few years for better fuel efficiency. Why can they do that, whereas can't we do the same with things like GNU coreutils?

> Remember, it took C many years to "catch on", and even longer to become the de facto standard for systems work. We can't rush this sort of thing... especially given the sheer magnitude of things depending on this code.

Actually, Bell Labs had no problems with building their entire system front to back in their new unproven language C instead of using Fortran. I'm glad they went the way they did.

Re: Cross-platform Rust rewrite of the GNU coreutils

#262

Post by jahewson »

jordigh wrote: post #214
mapleoin wrote: post #22

What are the licensing implications for this kind of work? I assume the authors used GNU coreutils as more than just inspiration. They probably read all the original code and reused some of the solutions (obviously ported to Rust). Shouldn't the derivative work still be covered by the GPL?

Reading GPL'ed code and reimplementing it makes a good case for why the rewrite should also be GPL'ed. I don't know if they did this here, but if so, they should GPL their version as well. For GNU Octave, we stress very strongly that anyone who has read Matlab's source code is ineligible to contribute to Octave. This is because, should it ever come down to it, we want to be able to ascertain that our implementation i…

Nope, just as reading non-GPL'd code and then writing a GPL'd version is legitimate, so too is reading GPL'd code and writing a non-GPL'd version. As long as you're not literally copying and pasting the code.

While you're free to invent any contribution rules you like for Octave, there's really no need for such drastic measures. It might give you a nice piece of mind, but it's not legally necessary - it's perfectly possible for someone who has read the Matlab source code to contribute to your project without copying anything. You'd just rather not have to think about it, which a is pragmatic, but heavy-handed restriction.

Re: Cross-platform Rust rewrite of the GNU coreutils

#263

Post by samth »

pcwalton wrote: post #138
catawbasam wrote: post #128

Earlier quoted context omitted.

'Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man.' This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier or naive about software quality is something I find profoundly mis-guided.

> This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. "Rust doesn't eliminate all bugs" is a rephrased version of "Some well-formed rust programs will generate machine code that behaves differently from what the programmer expected." > That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier…

I don't think he's making the argument you think he's making. This is clearer in his later comments in the thread, in which he says that UB is much scarier in systems like Fossil.

His argument is that achieving the level of quality that SQLite has requires verification (in a broad sense) after the compiler, and that's what he does. If you consider the goal to be producing quality-assured binaries, then you can treat UB, compiler bugs, and many other things as falling in a similar category, which are almost certainly eliminated by an MC/DC test suite.

As you say, this isn't feasible for almost any software (as John said in the blog post, SQLite is the only program he knows of that has MC/DC testing when not required by law). But it does mean that rewriting SQLite in Rust wouldn't provide as much value as rewriting many other things where the binaries do not have such guarantees.

Re: Cross-platform Rust rewrite of the GNU coreutils

#264

Post by andrewflnr »

weberc2 wrote: post #220
pcwalton wrote: post #95

Earlier quoted context omitted.

Disappointing to see Hipp make that argument. It's trivially refuted. Yes, all programming languages allow the programmer to write bugs. But languages very much vary in how many , and what kinds of bugs programmers write in practice . Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man. If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in…

> If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in the important areas in practice than Rust (plus kcov [1] if you like) does. I don't have a dog in this fight, but I don't see how the burden of proof is on Hipp rather than the folks proposing the change. In other words, shouldn't the "rewrite it in Rust" folks have to prove that the cost of their propos…

Sure, the primary burden of proof is on those proposing a change. However, any time you stand up and make an argument, the burden is on you to make sure it actually makes sense, and that goes for both sides.

Re: Cross-platform Rust rewrite of the GNU coreutils

#265

Post by wangchow »

nickik wrote: post #174
wangchow wrote: post #160

Earlier quoted context omitted.

Difficulty is relative. If you don't study modern C idioms your C code will be crap. Same goes for Rust. Honestly using higher-level languages is the same mentality as taking a pill to magically lose weight. It's quick, but detrimental (to programmers ability) in long term. Programming becomes easy but in the long term most people forget how algorithms and data structures work, in addition to cache mechanisms and oth…

I disagree. When programming C you have to remember so much stuff and be so carful about even the simplest things, this mental effirt takes from your resources, and I dont care how good a C programmer you are. The mental capacity saved can be invested in higher level design issues that gone get you a lot more in the long run.

I am not arguing that higher-level languages are worse and that everything should be coded in C. Each problem requires different tools for its solution. And to that end, high-level concepts should be done in a language-agnostic manner.

However, lower-level understanding is paramount. For instance, and most-importantly today, taking advantage of multi-threading requires understanding of cache-coherence, memory-alignments, et cetera.

Therefore, while a proper serial algorithm today may be correct its scalability is going to be limited without lower-level understanding. Although, I'll admit that can be built into the higher-level languages (like concurrency in Clojure for instance), but I personally prefer to understand what is going on rather than blissful ignorance :)

Re: Cross-platform Rust rewrite of the GNU coreutils

#266

Post by yberreby »

mikevin wrote: post #215
yberreby wrote: post #124

Earlier quoted context omitted.

Autocompletion: YouCompleteMe + racer + racerd Source formatting: rust.vim + rustfmt Automatic building: https://github.com/passcod/cargo-watch Error display: tmux, terminator or iTerm2 split planes Rust has pretty good tool support in vim and Atom, for example (can't speak for the rest). We don't have stellar IDE support yet, but it's on the to-do list: https://www.rust-lang.org/ides.html Personally, I don't miss ID…

Do you use all of the autocompletion options you listed? Or is YCM with --racer-completer enough?

I don't see another autocompletion option than YCM with --racer-completer in my list, so yes and yes. I use all the tools I listed regularly.

Re: Cross-platform Rust rewrite of the GNU coreutils

#267

Post by gillianseed »

ChristianBundy wrote: post #188
pnathan wrote: post #150

Earlier quoted context omitted.

I've submitted an issue asking to shift to a GPL license. My general perspective on code I write that isn't for work - it has to be GPL. I refuse to have my code be yoinked by random corporations for their profit without having the code shared downstream.

Like many other programmers, I avoid GPL'd code like the plague. The idea that you can own an idea seems ridiculous to me, and it feels unjust to sue "random corporations" for using ideas that you published. We're standing on the shoulders of giants, and I see the GPL as a tumor that's draining the world's resources. Just my two cents.

I find your reasoning very strange, if anything, GPL ensures that the 'ideas' are instead made available for anyone to use, provided that they in turn do the same.

If anything would be a 'tumor' by your reasoning around ideas, it would be proprietary software, which is what GPL prevents.

Re: Cross-platform Rust rewrite of the GNU coreutils

#268

Post by jahewson »

frutiger wrote: post #234
jordigh wrote: post #214

Earlier quoted context omitted.

Reading GPL'ed code and reimplementing it makes a good case for why the rewrite should also be GPL'ed. I don't know if they did this here, but if so, they should GPL their version as well. For GNU Octave, we stress very strongly that anyone who has read Matlab's source code is ineligible to contribute to Octave. This is because, should it ever come down to it, we want to be able to ascertain that our implementation i…

> This is because, should it ever come down to it, we want to be able to ascertain that our implementation is completely original I appreciate that this is how it works today, but isn't that a completely outrageous idea? A well read, well travelled person will have seen countless things that will influence their future behaviours. It is not uncommon to completely forget a particular source of inspiration (sometimes w…

Yes, it is. This is an excessive level of caution which does not reflect the permissiveness of actual copyright law. However, there is the concept of "unintentional copying", which is effectively what you're describing, and which is still considered an infringement under copyright law, however it's very, very unlikely that such an argument would be used in court with regards to software, and even more unlikely that a jury would be convinced by it. It's really more of an argument for subjective and creative endeavours, such as hearing a music clip or seeing a logo design or piece of art, which leaves a strong impression.

Lawyers tend to fuel the FUD around this with "best practice" concepts such as "clean room", which is a drastic overreaction. It's akin to saying "if you want to be an author you should never read any books, in case you accidentally copy one of them". Sigh.

Re: Cross-platform Rust rewrite of the GNU coreutils

#269

Post by Alupis »

pcwalton wrote: post #261
Alupis wrote: post #258

Earlier quoted context omitted.

> We think of ourselves as one of the most fast-moving industries We're fast-moving because our foundation is solid and not changing (ie. CoreUtils and gang). It's an assumption that these things "just work" with zero fuss and weirdness between systems. We build on-top of these systems, so changing them out from underneath us all is a dramatic shift. Perhaps Rust is the key to making these things better. I never clai…

> We're fast-moving because our foundation is solid and not changing (ie. CoreUtils and gang). That isn't a definition of "fast moving" that I would use. It sounds like slow moving. Why is innovation in the core layers of the system less legitimate than innovation in social media apps? Boeing has no problem upgrading their engines every few years for better fuel efficiency. Why can they do that, whereas can't we do t…

> That isn't a definition of "fast moving" that I would use. It sounds like slow moving.

Perhaps I didn't word it correctly.

When you want to write the next WhatsApp, you don't have to start from scratch. The OS has been taken care of for you, and you can expect it to "just work".

This solid foundation allows innovation to build on-top, at a rapid pace. If your foundation was constantly changing, you'd have to account for all sorts of weird intricacies, non-portable code, etc (like the old days).

A rapidly changing OS isn't really what you want for a production environment. In fact, you want it to be as constant as possible, so that you are free to do your work.

> Actually, Bell Labs had no problems with building their entire system front to back in their new unproven language C instead of using Fortran

You're right. But do remember it took a long time for it to propagate. Today, there's still systems not written in C (although they are in the minority). At the time, a lot of systems were written in pure assembler, and it took a long time to convince those guys that C was ready as a replacement for most tasks (and C changed dramatically in that time period).

I'm glad they went the way they did at the time -- but today with all these things built on top (financial markets, governments, big mega-corps, small ma n' pa shops, etc...), we need slower changes in order to keep the stability.

Re: Cross-platform Rust rewrite of the GNU coreutils

#270

Post by peterwwillis »

masklinn wrote: post #157
peterwwillis wrote: post #112

Earlier quoted context omitted.

The only operating systems that are certified to conform to POSIX are old Unix operating systems. GNU has always been non-POSIX. And honestly, a POSIX implementation would be more trouble than it's worth. The point of coreutils is to have utilities that make up the ability to write scripts for and interact with your operating system, right? Well, what operating system?? A POSIX-compliant one? Or just a mostly-POSIX-c…

That's not a question of certification or extensions, I'm talking about being able to write portable scripts . Due to the interaction between its definition and its extensions, echo is a prime example of being impossible to use portably (except in the very restricted case of only literal strings without escapes which don't start with a -). > Ultimately, each platform has quirks, and it is up to the developer to port…

Ok, I think I understand the confusion now. You seem to be of the impression that shell scripts are like bytecode executed in a virtual machine. That is the only way I know of that you could write an application for a platform that doesn't exist and expect it to work. But even for that to work, it would need to be the same VM, and bytecode generated by & for it, or there's still no guarantee it will work.

Of course, you can already write a shell script for a particular shell, distribute that shell to that system, and depend on the shell to properly execute your script [by using internal functions only]. But that defeats the whole purpose of following a standard like POSIX, or caring at all how any given platform's 'echo' program works.

Bottom line, though: two independent implementations of a standard provide no guarantee they will work together. Practice over a couple decades shows this to be the case.

Post reply on HN