Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cross-platform Rust rewrite of the GNU coreutils

github.com

Post reply on HN
Print view View original

111–120 of 498 posts

Re: Cross-platform Rust rewrite of the GNU coreutils

#111

Post by thedevil »

mapleoin wrote: post #22

What are the licensing implications for this kind of work? I assume the authors used GNU coreutils as more than just inspiration. They probably read all the original code and reused some of the solutions (obviously ported to Rust). Shouldn't the derivative work still be covered by the GPL?

Does anyone have more solid info on whether the GPL would apply in this case?

I would really like to know if licensing agreements (and copyrights) still apply to ported code (assuming there's no patents on the algorithm).

Re: Cross-platform Rust rewrite of the GNU coreutils

#112

Post by peterwwillis »

masklinn wrote: post #36
gjem97 wrote: post #14

Earlier quoted context omitted.

Ok, I'll bite. What's wrong with "echo -n"?

http://www.in-ulm.de/~mascheck/various/echo+printf/ is the raw data version. POSIX defines echo as only taking string parameters and no options, but notes that behaviour facing `-n` is implementation-defined. BSD and GNU echo implement `echo -n` as not printing a trailing newline, but `echo` commonly calls to a shell builtin which may or may not follow that behaviour (and may switch behaviour depending on whether the…

The only operating systems that are certified to conform to POSIX are old Unix operating systems. GNU has always been non-POSIX. And honestly, a POSIX implementation would be more trouble than it's worth.

The point of coreutils is to have utilities that make up the ability to write scripts for and interact with your operating system, right? Well, what operating system?? A POSIX-compliant one? Or just a mostly-POSIX-compliant one? Or one with POSIX extensions? How would your utilities know the difference? How would the OS know how to deal with these utilities? Would the user know the difference?

Ultimately, each platform has quirks, and it is up to the developer to port and test their script or application to a platform and make any necessary changes. This extends to far more than just POSIX compliance.

Re: Cross-platform Rust rewrite of the GNU coreutils

#113

Post by Alupis »

thegeomaster wrote: post #12
ignoramous wrote: post #8

Rust is a great choice for systems programming it'd seem. Esp considering that a well-tested, battle-hardened code-base like SQLite faces problems [1] solely due to the nature of the language its written in. [1] https://news.ycombinator.com/item?id=11312918

Not sure why you're being downvoted. I don't see why we shouldn't be moving to languages like Rust given the chance, as C makes it far more difficult to write safe and correct code.

> I don't see why we shouldn't be moving to languages like Rust given the chance, as C makes it far more difficult to write safe and correct code

I agree in principal, however the timing is wrong.

Rust is a very new language (less than 6 years old), and is undeniably totally unproven. It is the latest "buzz" language, and may not be around long term... nobody knows.

What-more, a full-fledged re-write of CoreUtils in Rust is unlikely to be used by any production environment, because this new CoreUtils will also be undeniably totally unproven. This is the same growing pains LibreSSL have been experiencing, lots of gung-ho fans, but very few actual users (outside of OpenBSD/FreeBSD) - and their undertaking is arguably a lot easier, since they're just cleaning a codebase, not starting from scratch.

Average users who just consume distro's are not going to switch to a new unproven CoreUtils (even if they knew how), and distro maintainers are not going to switch until it's proven either. It will take a huge company with a huge install-base switching and testing it in production for many years before others start to feel comfortable... however this is also an enormous burden on said mega-corporation, for little-to-zero perceived benefits.

Yes, in principal, it's "safer" code, but to a mega-corp with thousands of installs, the risk is too great. New bugs, language pitfalls, behavior potentially changing, etc. Perhaps Rust dies, perhaps it's replaced with an even better alternative. It will take a LOT of time to work all this out.

Flatly, re-writing a several decade's old, matured codebase in today's flavor-of-the-week language is not a good idea. It's a waste of time and effort.

Let the languages mature more, do more systems work that doesn't involve replacing the foundation we all stand on... and maybe, in 5-10 additional years, we'll see where Rust goes.

Re: Cross-platform Rust rewrite of the GNU coreutils

#114

Post by binarycrusader »

jblow wrote: post #41
steveklabnik wrote: post #34

Earlier quoted context omitted.

It looks like there are 134 instances of 'unsafe', in 23,000 lines of Rust. And it looks like a lot of that unsafe is to FFI into libc. Rust works with GDB, so you end up debugging like anything else. IDE integration is being actively worked on, and sorta-kinda works in my understanding.

What is the roadmap on getting rid of the need for libc? Given how terrible libc is, I personally would make that a high priority, though I guess in Linux you can't even start up a process without libc (maybe that is a misunderstanding?), which makes the situation less clean, but at least you could get to a point where you never call back into it after entry into main.

Realistically? Never. If you're running on a UNIX system, libc is your only portable interface to the system. You can use the syscall layer on some NIX-like platforms (such as Linux), but it will leave you to replicate a lot of the work that it already does.

On Solaris, as one example, there is no stable syscall layer -- libc is your only interface to the system. There's good reason for that too; on Solaris, libc is updated with security and performance improvements on a regular basis for each new hardware generation and other improvements in the operating system. It automatically accounts for things that might cause pipeline stalls (think memcpy, etc.) in newer processor generations, and so on.

Now with that said, what you could advocate is that libc, etc. be rewritten in RUST, but exposed via the "C" ABI convention that RUST provides. It wouldn't be as great as native RUST, but it would still be an improvement.

There are no compelling arguments for getting rid of libc that I've heard yet for platforms where libc is well-maintained. On Solaris, there are strict compatibility guarantees for every interface, so applications can assume changes in libc will never break them as long as the interface they're using lists an appropriate stability level in the manual page.

A world where every language implements its own version of libc seems like it would increase the number of defects, not decrease them. We could of course argue that those implementations might be better than libc, but I think whatever possible benefit there might be there is lost in the likelihood that each language will have its own flaws in its implementation.

I believe the best view of an operating system is as a fully-integrated set of components. Everything from the kernel up to the system libraries should look and act consistently, and that's impractical to do if every component is viewed as interchangeable. Integration from top to bottom in an OS stack can produce incredibly great results and provide unparalleled reliability, availability, and performance.

Re: Cross-platform Rust rewrite of the GNU coreutils

#115

Post by Jtsummers »

catawbasam wrote: post #102
kbenson wrote: post #76

Earlier quoted context omitted.

Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate whatever machine code it wants without it being called a compiler bug. That’s an important finding. But as it happens, no modern compilers that we know of actually interpret any of the SQLite source code in…

"while ignoring that it's caused many problems already" Really? I believe Mr. Hipp is claiming that it hasn't. Do you have evidence to the contrary? I don't think the argument is that these cases should be ignored -- they are now corrected, after all. It is that most of these cases should be treated as low priority compared to issues that are creating observable problems.

I take kbenson to be saying that what Mr. Hipp claims isn't a problem for SQLite is or has been a problem for other projects. That is, other projects have been bitten by relying on undefined behavior (such as what a variable's initial value might be) that may have been consistent across a variety of implementations, and then suddenly wasn't. Perhaps it's safe to say that today we don't need to address these issues on this project in this language, but that's taking on technical debt and setting ourselves up for work in the future (that may be difficult to identify at that time).

Re: Cross-platform Rust rewrite of the GNU coreutils

#116

Post by pcwalton »

jfoutz wrote: post #103
masklinn wrote: post #39

Earlier quoted context omitted.

I expect it's because because the first part is essentially preaching to the choir, and because Richard Hipp very much disagrees with the second part[0] > Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt. (see link for expansion on that matter, which is a question of tooling and testing) [0] http://blog.regehr.org/archives/1292#comment-18452

The quality of tooling, and the ability of experts to verify the output of machine code is a really important point. I think I'd agree, rust at this point would hold back an elite developer like Richard Hipp. The promise of rust, which may or may not be realized is pushing some very common problems down to the compiler. All code has bugs, so the compiler probably does things wrong in some cases. As the tools mature,…

I think the idea that "elite" developers can write bug-free C (or even just network-facing C free of security-sensitive memory safety problems) is pretty well refuted at this point. Now you can write bug-free C if you're willing to spend enormous time and money on testing: this is to a first approximation what SQLite did. But that only makes economic sense for a small minority of projects. Just putting "elite" developers on a C project, without the huge verification cost, is by itself not enough to eliminate bugs, as much as we as hackers would like to think it is.

(The one exception may be, like, DJB. But djbdns/qmail are very unusual C programs in many ways.)

Re: Cross-platform Rust rewrite of the GNU coreutils

#117

Post by kbenson »

catawbasam wrote: post #102
kbenson wrote: post #76

Earlier quoted context omitted.

Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate whatever machine code it wants without it being called a compiler bug. That’s an important finding. But as it happens, no modern compilers that we know of actually interpret any of the SQLite source code in…

"while ignoring that it's caused many problems already" Really? I believe Mr. Hipp is claiming that it hasn't. Do you have evidence to the contrary? I don't think the argument is that these cases should be ignored -- they are now corrected, after all. It is that most of these cases should be treated as low priority compared to issues that are creating observable problems.

I was making a general point using his statement as the impetus, not a specific point in the case of SQLite. Undefined behavior (specifically, developers relying on it being consistent) has caused many problems in the past, and will undoubtedly do so in the future. Relying on behavior which by it's definition is impossible to rely on, is not a situation I think we should be defending.

Re: Cross-platform Rust rewrite of the GNU coreutils

#118

Post by steveklabnik »

haberman wrote: post #110
quotemstr wrote: post #99

Earlier quoted context omitted.

Rust is a terrible choice. Its standard library assumes malloc never fails.

This is downvoted, but I'd rather see it refuted. Is this false? Is SQLite as written tolerant of malloc() failure?

The Rust standard library does assume that allocation succeeds. The language itself knows nothing about the heap, and so you can write allocators that do whatever you wish. Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure. I am less knowledgeable about OSX and Windows.

I cannot speak to SQlite in this regard.

Re: Cross-platform Rust rewrite of the GNU coreutils

#119

Post by heavenlyhash »

_pfxa wrote: post #67

> Many GNU, Linux and other utils are pretty awesome, and obviously some effort has been spent in the past to port them to Windows. However, those projects are either old , abandoned, hosted on CVS , written in platform-specific C, etc. I have seen such a paragraph in another projects README, IIRC a Go rewrite of standard utilities. I do not understand why a project would be obsolete because it's on CVS or is old. CV…

Open source is useless if the majority of people in the open community -- for whatever reason -- can't build it.

If a person can't build the thing, they effectively have no power to contribute to changing it. If they have no power to contribute to changing it, it's... spiritually, missing the most essential parts of open source.

This is the fundamental issue driving comments like the quote above.

You can tell me that not having a dependency management system, not having a sane version control system, using "some" C compiler (sans a matrix of tests specifying a range of expected good compilers), arbitrarily fine-grained platform specificness meaning an average OSS contributor can never reasonably test their changes against all targets..... all of these things can be "worked around". But at some point, the litany of issues -- some of which take a new contributor dozens of hours to work around -- becomes a simply overwhelming barrier to contribution.

It's time to admit that a foundation of workarounds in FOSS development processes at the very core of our systems is a problem.

Re-writing it in language $x may not be the solution, but it's certainly understandable that there's a widespread desire for simply getting better toolchains underneath our most basic essential systems.

Re: Cross-platform Rust rewrite of the GNU coreutils

#120

Post by hobarrera »

gravypod wrote: post #64
theonewolf wrote: post #24

Earlier quoted context omitted.

I don't understand why you need an IDE, I believe Rust is simple enough that you can work on things in Notepad++ or Vim or Emacs.

Here are the features I like in an IDE that make be very productive: - Autocompletion - Mass rename - Source formatting - Integrated debugger interface with breakpoint insertion and overlying of state on source - Integrated VCS control - Automated deploy - Error display - Automatic importing of modules - Source cleanup (Automatic loop transformation) - Automatically building my project That's just a few things that I…

vim (+ sh) cover all those things, except maybe the last two.
Post reply on HN