Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cross-platform Rust rewrite of the GNU coreutils

github.com

Post reply on HN
Print view View original

421–430 of 498 posts

Re: Cross-platform Rust rewrite of the GNU coreutils

#421

Post by FireBeyond »

pnathan wrote: post #150
coldpie wrote: post #83

Earlier quoted context omitted.

I've contributed to this project, and yeah, this was a major concern for me while I was doing it. If it shared a license with GNU coreutils, then code sharing would be free and the project would be built much faster because I could just use coreutils's algorithms. As it is, I haven't done any real, hard work for it because frankly, I won't want to re-invent that wheel. The project isn't terribly far along. I wonder i…

I've submitted an issue asking to shift to a GPL license. My general perspective on code I write that isn't for work - it has to be GPL. I refuse to have my code be yoinked by random corporations for their profit without having the code shared downstream.

And sadly the holy war begins...

arguments over derivatives, clean room implementations...

something yanking out copyright notices to replace with "this code is now GPL"...

Re: Cross-platform Rust rewrite of the GNU coreutils

#422

Post by greggyb »

cname wrote: post #247
ChristianBundy wrote: post #188

Earlier quoted context omitted.

Like many other programmers, I avoid GPL'd code like the plague. The idea that you can own an idea seems ridiculous to me, and it feels unjust to sue "random corporations" for using ideas that you published. We're standing on the shoulders of giants, and I see the GPL as a tumor that's draining the world's resources. Just my two cents.

What on earth does the GPL have to do with owning _ideas_? Please explain. Also, do you think that Linux, for example, is draining the world's resources? If so, how?

If your code is GPLed you impose a cost on someone else to use it. Whether or not that cost is morally justified is not the point of the objection. I am making no judgment as to the righteousness of the goal of using a GPL or similar license.

By using a GPL you strictly limit who can consume your code to those who are willing to be bound by your views on what is right.

In this way (and limited explicitly to the scope of topics I am directly addressing in this comment), GPL licensing is similar to proprietary licensing: limits are imposed upon consumers of your code based on your licensing decision.

Again, I am not making a moral judgment or normative statement about what is good or right when it comes to code reuse or copyright in general.

Licenses which do not impose requirements upon consumers of the product are more free in the sense of allowing more behaviors, or placing fewer restrictions.

To me the key distinction is one of code freedom vs human freedom. Is the code free from malicious tampering or is the human free to make decisions and take action unhampered by another?

If I may quote the late Milton Friedman, "Heaven preserve us from the sincere reformer who knows what's good for you and by heaven he's going to make you do it whether you want to or not. That's when the greatest harm is done."

P.S. This is not an observation about you, because I don't know you, but I think we can observe the amount of hedging and defensive posturing I've taken by default in this response is illustrative of the type of response I can expect, based on observing similar discussions across various fora online. I hope that in this case that defensiveness was not necessary (:

Re: Cross-platform Rust rewrite of the GNU coreutils

#423

Post by jerf »

Dylan16807 wrote: post #407
pcwalton wrote: post #386

Earlier quoted context omitted.

> What parts of the STL can be faster by treating uninitialized variables as impossible? What I'm mostly thinking of is allowing unused branches to be pruned. The STL tends to get inlined really heavily, which results in a whole pile of IR being emitted for what look like very simple operations. Based on the actual parameters and state, the optimizer then wants to prune out as much dead code as possible to reduce i-c…

I still don't understand. Why would length be undefined if that's how you tell whether a string is small or not? Even if you can remove one of the branches because you know if the string is small, the logic of "this branch can't happen" -> undefined -> delete sounds more complex than "this branch can't happen" -> delete.

The order is undefined -> "I, the compiler, declare this branch can't happen" -> delete. The middle step is valid because "undefined behavior" permits the compiler to make that declaration, then act on it. If you don't want it to do that, use defined behaviors only, which is a great deal easier said than done. Partially because of how hard it is to avoid it in your own code, and partially because it is shot through all the other library code (which as pcwalton points out, courtesy of aggressive inlining, is also your code).

I was skeptical about all this about six months ago myself, but the continuous stream of articles on this topic, plus the spectacular and highly educational failure of Friendly C (and not just that it failed, but why it failed, which is why I posted that exact link) has satisfied me. It is also part of why I've stepped up my own anti-C rhetoric since I've been so convinced... as bad as I thought C was, it really is, no sarcasm, not merely "hating", even worse than I thought. I am honestly scared to use it professionally and pretty much refuse to touch it without good static analysis support.

Re: Cross-platform Rust rewrite of the GNU coreutils

#424

Post by spiralpolitik »

icebraining wrote: post #415
spaceheeder wrote: post #242

Earlier quoted context omitted.

From the README: "These are based on BSD coreutils rather than GNU coreutils as these tools should be as minimal as possible." If all of these utilities are running in userspace, per Redox's microkernel architecture, then what is the advantage of intentionally not making them feature-rich?

It has the advantages that come from following the UNIX style. To learn more, I recommend reading this short paper by Rob Pike and Brian Kernighan, published in 1984: http://harmful.cat-v.org/cat-v/unix_prog_design.pdf

"Those days are dead and gone and the eulogy was delivered by Perl" - Rob Pike (2004)

Re: Cross-platform Rust rewrite of the GNU coreutils

#425

Post by collinmanderson »

pcwalton wrote: post #95
masklinn wrote: post #39

Earlier quoted context omitted.

I expect it's because because the first part is essentially preaching to the choir, and because Richard Hipp very much disagrees with the second part[0] > Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt. (see link for expansion on that matter, which is a question of tooling and testing) [0] http://blog.regehr.org/archives/1292#comment-18452

Disappointing to see Hipp make that argument. It's trivially refuted. Yes, all programming languages allow the programmer to write bugs. But languages very much vary in how many , and what kinds of bugs programmers write in practice . Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man. If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in…

I'm new to Rust. It is possible for a program written in C to link to a library written in Rust?

Is it theoretically possible to rewrite Sqlite in Rust and still maintain compatibility with programs written in C?

Re: Cross-platform Rust rewrite of the GNU coreutils

#426

Post by pcwalton »

Dylan16807 wrote: post #407
pcwalton wrote: post #386

Earlier quoted context omitted.

> What parts of the STL can be faster by treating uninitialized variables as impossible? What I'm mostly thinking of is allowing unused branches to be pruned. The STL tends to get inlined really heavily, which results in a whole pile of IR being emitted for what look like very simple operations. Based on the actual parameters and state, the optimizer then wants to prune out as much dead code as possible to reduce i-c…

I still don't understand. Why would length be undefined if that's how you tell whether a string is small or not? Even if you can remove one of the branches because you know if the string is small, the logic of "this branch can't happen" -> undefined -> delete sounds more complex than "this branch can't happen" -> delete.

LLVM has been doing this optimization since 2008, and it contains justification in the commit message. LLVM commit #60470:

    Teach jump threading some more simple tricks:
    
    1) have it fold "br undef", which does occur with
       surprising frequency as jump threading iterates.
    ...
Chris didn't cite any specific numbers in the log, but I believe him when he says it actually happens. You should be able to run "opt -O2 -debug-only=jump-threading" and see where it does :)

Related, with some actual numbers to prove it helps, LLVM commit #138618:

    SimplifyCFG: If we have a PHI node that can evaluate to NULL and do a load or
    store to the address returned by the PHI node then we can consider this
    incoming value as dead and remove the edge pointing there, unless there are
    instructions that can affect control flow executed in between.
        
    In theory this could be extended to other instructions, eg. division by zero,
    but it's likely that it will "miscompile" some code because people depend on
    div by zero not trapping. NULL pointer dereference usually leads to a crash so
    we should be on the safe side.
        
    This shrinks the size of a Release clang by 16k on x86_64.

Re: Cross-platform Rust rewrite of the GNU coreutils

#427

Post by kazinator »

dikaiosune wrote: post #159
kazinator wrote: post #104

Complete rubbish: Many GNU, Linux and other utils are pretty awesome, and obviously some effort has been spent in the past to port them to Windows. However, those projects are either old, abandoned, hosted on CVS, written in platform-specific C, etc. Rust provides a good, platform-agnostic way of writing systems utils that are easy to compile anywhere, and this is as good a way as any to try and learn it. Cygwin has…

Is the cygwin port just some different build files, or is it genuinely different code? A Rust codebase probably wouldn't (in most cases) need to be modified to run on different supported platforms.

Can you show me a Rust program that calls CoCreateInstanceEx on Windows, ported to Solaris?

Re: Cross-platform Rust rewrite of the GNU coreutils

#428

Post by dikaiosune »

kazinator wrote: post #427
dikaiosune wrote: post #159

Earlier quoted context omitted.

Is the cygwin port just some different build files, or is it genuinely different code? A Rust codebase probably wouldn't (in most cases) need to be modified to run on different supported platforms.

Can you show me a Rust program that calls CoCreateInstanceEx on Windows, ported to Solaris?

Disclaimer: I don't really speak developer Windows.

Can you show me a Rust program that calls CoCreateInstanceEx at all? Also, what is that for? The MSDN page isn't particularly helpful.

Re: Cross-platform Rust rewrite of the GNU coreutils

#429

Post by pcwalton »

AnimalMuppet wrote: post #397
pcwalton wrote: post #375

Earlier quoted context omitted.

> Examples? I'd be surprised if best practice C++ (C++11, say) had memory safety issues. Pwn2Own last week. For more examples, search any browser engine's bug tracker. Yes, this is all modern C++.

> Yes, this is all modern C++. Is it? You're telling me that all code in all browsers have been re-written into C++11 (or C++14) with best practices? I don't believe you. At a minimum, I'm going to need some documentation before I believe that. [Edit: I'm not trying to pull a No True Scotsman here. I just doubt that browsers have been completely rewritten into modern C++, or with anything approaching best practices.…

Most exploits tend to be in new code (contrary to popular belief), which in all modern browsers is written in modern C++. The WTF (Blink/WebKit) and the MFBT (Firefox) are state-of-the-art template libraries; you are free to search for those libraries and verify for yourself. New C++11 features such as rvalue references do nothing to avoid memory safety problems; in fact, they make them worse, since "use-after-move" is now a problem whereas it wasn't before.

I know it's hard to believe, but C++ is not memory safe, old C++ or modern C++, in theory or in practice. The new C++ features do effectively nothing to change this. As far as use-after-free goes, C++ basically adds safety over C in two places: (1) reference counting is easier to use and is easier to get right; (2) smart pointers are arguably somewhat less likely to get freed before accessed again due to the destructor rules (though I think (2) may not be true in practice). Browsers have been making use of these two features for a very long time.

Bringing up modern C++ here is "no true Scotsman" unless you can point to a specific C++11 feature that browsers are not using that is a comprehensive solution to the use-after-free vulnerabilities they suffer from. There is no such feature I am aware of.

Re: Cross-platform Rust rewrite of the GNU coreutils

#430

Post by paulmlewis »

haberman wrote: post #85
masklinn wrote: post #39

Earlier quoted context omitted.

I expect it's because because the first part is essentially preaching to the choir, and because Richard Hipp very much disagrees with the second part[0] > Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt. (see link for expansion on that matter, which is a question of tooling and testing) [0] http://blog.regehr.org/archives/1292#comment-18452

He certainly has a point, but I think he goes too far in assurances about the current sqlite3.so. Testing "every single instruction" does not guarantee that all C-level UB has been eliminated, because some bugs can be input-dependent. For example, even if your coverage tells you that this function has been tested, it could still trigger undefined behavior for other inputs that trigger overflow: int f(int x) { return…

I ask this from a position of seeking knowledge rather than being adversarial, the discussions on undefined behaviour from a C language perspective always seem muddled to me between what's undefined at the C language level such that a compiler might take advantage of it and produce results that would be unexpected looking at the code, vs. the machine code generated faithfully represents the language statements but a bug may be present when processing some external input at runtime.

If the inputs here are coming from something external to the program, such that the compiler can't know the value of x then the machine code should be faithful to the language statements, it's a bug if overflow occurs which may have other runtime implications, but the compiler isn't going to remove some chunk of code, etc. because of it.

From their testing page SQLite say they also test boundary conditions (I don't know to what granularity that is though) which which may catch something like this, which is agreeing that instruction level coverage isn't enough. They also run their tests with all the various sanitisers enabled.

Isn't this the same case in Rust, the non-release builds would need to see a suitable test case for the overflow check to cause a panic.

I guess I'm asking is this example relevant to concerns about undefined behaviour and optimising compilers vs. Rust? Since the function is input value dependent in practice isn't this more like implementation defined behaviour at runtime, in that a platform will alway provide a consistent behaviour, e.g. overflow, trap, saturate, etc.?

I've followed regehr's blog for a few years, I've read a lot about Rust, but I'm mostly working in higher level dynamic languages and don't have a lot of hands on experience with C or Rust and just wondering if I'm missing some subtlety here.

Post reply on HN