Earlier quoted context omitted.
C is not a particularly low-level. It has no (standard) way to control vectorization, stack usage, calling conventions, etc. It just had an insane about of money spent on making its compilers optimize better.
You are right, but no language has tried to claim the space of giving the user more control, so C remains the lowest level language we have that is portable. I think there are lots of opportunities in this space, but i don't know of anyone working on it.
Memory safety is necessary, not sufficient
131–140 of 162 posts
Re: Memory safety is necessary, not sufficient
#132Earlier quoted context omitted.
> If Rust doesn't give you more and better support in doing that than C does, then it's not really making a difference. It does and if you've been missing that you've misunderstood this whole discussion. Rust allows you to wrap up some unsafe code in a safe abstraction. You use the type checker to enforce your invariants, such that the unsafe code can be reviewed in isolation. Rust gives you exactly what you're sayin…
> It does and if you've been missing that you've misunderstood this whole discussion. Don't tell it to me, tell it to the person I was replying to.
> any nontrivial program will have unsafe somewhere, and unsafe can cause failures arbitrarily far away from the incorrect code
But the point I was trying to make is that this claim is facially true but misses the critical context: in idiomatic Rust, the unsafe keyword is not the barrier that enforces integrity of the system.
The integrity of the system is enforced by the type checker, the same way it always is. The unsafe annotation alerts the reader that there exists an invariant the compiler can't check. Idiomatic code will have a SAFETY comment above describing the invariant. It should be locally possible to reason about how the type abstraction used to encapaulate truly enforces that invariant.
This is what Rust people are talking about when we say wrap unsafe code in a safe abstraction. If we do it right, it's no longer possible for spooky action at a distance.
Now you might go and point out the fallibility of humans and all that, and you'd be right, but that's in fact what makes it so valuable to try.
Re: Memory safety is necessary, not sufficient
#133Earlier quoted context omitted.
Undefined behavior is critical for performance. Without undefined behavior, C compilers would not be able to optimize at all. You'd be running everything at -O0 or worse.
> Undefined behavior is critical for performance Not only is this not true, it's trivially easy to prove it's not true. Both rustc and clang generate LLVM IR and use LLVM to optimise and generate machine code. The code that's generated is equally performant, as you'd expect since most of the optimisation is being done by LLVM, not the front end. The difference between the two frontends is that rustc is stricter, reje…
Re: Memory safety is necessary, not sufficient
#134Earlier quoted context omitted.
> If you at all break the rather subtle rules, then essentially anything can and will happen. If by subtle rules you mean your invariants, that is missing fundamental assumptions. It's akin to making a building without foundation and load bearing structures. > So for example, there was recently a thread where someone had code that checked if a value was in range to safely coerce it directly to an enum then did so. Bu…
> If by subtle rules you mean your invariants, that is missing fundamental assumptions. > It's akin to making a building without foundation and load bearing structures. That's exactly how C approaches UB too. > However note the UB goes away if you never use any unsafe code. Or if you expand your unsafe to encompas some safe code. Right. The problem is that's untenable; any nontrivial program will have unsafe somewher…
> That's exactly how C approaches UB too.
In theory yes. In practice it's been proven that's not what happens. > The problem is that's untenable; any nontrivial program will have unsafe somewhere, and unsafe can cause failures arbitrarily far away from the incorrect code.
Not really. Ignoring any and all non-compiler tooling, UB in code can be traced to:
A) unsafe region in code where its invariants are invalidated
B) if no unsafe code exists in project, then an unsoundness bug in the Rust standard lib.Safe code needs to be safe for any possible value of arguments, fields, variables. Doing otherwise is unsoundness bug.
In C, the code you need to look is essentially ALL of your code + C std lib. In Rust, you can focus on places where safe and unsafe code mingle. And IFF (if and only iff) your code contains 0 unsafe, can you be sure there is a Rust bug.
This however assumes any code.
Idiomatic code in Rust will treat unsafe with big large friendly letters `SAFETY: this works while X and Y hold`. How do you do this is C? Since most lines are possible source of UB do you annotate every line of code? I assume not. How do you ensure something will not be mutated? How do you ensure something is thread-safe? Do you use type bounds? And if not how?
> If Rust doesn't give you more and better support in doing that than C does, then it's not really making a difference.
Also that's the Nirvana fallacy, Rust doesn't have to be perfect and prevent EVERY undefined behavior forever. If it's improvement over the niche it's targeting (C/C++) then it's an improvement. I.e. is it harder to find UB in Rust or in C?Like you don't go around and say, "Well seatbelts don't prevent head injuries when you hit a wall head-on. So remove seatbelts!", you add airbags to cushion the blow.
And spoiler alert: It's giving you the airbags as well. The normal thing when encountering a UB isn't to try to track it down it's to run `miri`, it found UB as soon as it ran.
Re: Memory safety is necessary, not sufficient
#135I think it's underappreciated that Rust's `unsafe{}` doesn't exist in isolation. Rust has facilities for building safe abstractions on top of it, and has a culture of taking this abstraction layer seriously. Danger of unsafe features and FFI is usually conditional — you can use a pointer only until some point, or only on a single thread, etc. A use of unsafe in Rust doesn't become "be careful!" kryptonite spreading a…
> They don't have "you can use it only within this scope" (they may use a closure/callback to give access to an unsafe object, but these aren't hermetic, so that's a convention not a guarantee). You can use the same trick as Haskell's STRef to prevent reusing a "leaked" unsafe object, although it's cumbersome enough in Java that you may not want to.
Re: Memory safety is necessary, not sufficient
#136Our programs are growing so big by having so many (indirect) dependencies that we need a way to sandbox the libraries that we include from our main programs. This is the type of safety that I'm looking for, really.
What if the only way to access the file system was to call methods on an object that was provided to main()? Then libraries would only be able to access the file system if they got a reference to that object.
Re: Memory safety is necessary, not sufficient
#137> threads.into_iter().for_each(|t| t.join().unwrap()); What a great title for the almost-last section, it made me laugh. When I saw the title of the final section, I fell from my chair: > int atexit(void (*func)(void))
Re: Memory safety is necessary, not sufficient
#138> coming legislation around MSLs for government procuremen What’s he talking about here?
Re: Memory safety is necessary, not sufficient
#139Earlier quoted context omitted.
> People can (and do) point at the C spec for fault of this and it is true that if the C spec was more strict then these compilers would not have the free pass to do these crazy miscompilations. However there is nothing stopping these compilers from just not doing that, there is nothing stopping them from just defining their own sane behavior for what the C spec defines as undefined behavior. It is no longer the case…
> Your code can no longer be compiled by a compiler that does not support these special C dialects. What compiler might that be? It's true that MSVC doesn't have an equivalent of -fno-strict-aliasing, but that's because it just doesn't apply optimizations that assume strict aliasing in the first place. Admittedly the picture around signed overflow is more complicated, but it's essentially the same story. > Using thes…
Of course it's fine if the compiler doesn't implement this as an option, but rather as the default behavior -- as long as this is actually a documented guarantee, rather than "we just haven't implemented those optimizations yet, we might start exploiting this UB at any point in the future". I'm not familiar with what guarantees MSVC documents in this area.
> By that standard the Linux kernel isn't written in C.
I think that in a very real sense, it isn't. The kernel is written in standard C plus a few hundred GCC extensions, of which additional compiler options are but a small part. It took very extensive effort to make the kernel compile with clang (which already was mostly gcc-compatible, but nowhere near enough for the kernel's level of extension use).
> Even though Postgres compiles when -fwrapv and -fno-strict-aliasing are removed, and still passes all tests.
I think there is a lot of difference between "requires signed integer overflow to be well-defined because we explicitly rely on it" and "disable optimization based on signed integer overflow as a hardening measure, in case we got this wrong somewhere".
> The implication of what you're saying seems to be that all of these open source projects each independently decided to "go there own way".
You kind of make it sound like use of these options is common. To the best of my knowledge, this is not the case, and these projects are rare exceptions, not the rule. So, yes, they decided to go their own way (which is perfectly fine.)
Re: Memory safety is necessary, not sufficient
#140Earlier quoted context omitted.
>it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. This is true and when your compiler actually abides by these values it is shocking how many issues just go away. The problem is that gcc and clang are nowhere near a simple implementation, asking the question of how exactly gcc or clang arrived at some given assembly for some giv…
With all due respect, these “adversial compiler” expression just makes zero sense, and takes a lot away from your comment. Guess what, the world is complex, and software has no bound for complexity. Which is better, a multi-million lines compiler that hundreds of people worked on for decades, or a toy one in a couple thousand lines written by a single programmer? What if the former can create 2-10x faster code than t…
I was describing how it seems these complex compilers look for excuses to give you miscompilations, this "looking for gotcha's" makes the relationship appear adversarial to me.
>Guess what, the world is complex, and software has no bound for complexity.
My argument is that it really ought to. I think there is a diminishing return.
>What if the former can create 2-10x faster code than the latter (I probably even underestimate it, loop unswitching, vectorization, etc. can account for even more differences).
In code I've used with both compilers this has not been the case. Good profiling tools and manual hotspot optimization go a really long way.
>It turns out that we can build abstractions on top of abstractions, and if it’s designed well, it will scale with complexity
GCC has not scaled with complexity well enough, that's why we have people who lament its behavior.
>Would you change back to an OS that didn’t handle multithreading as it’s too complicated?
Concurrent programming makes problems easier, not harder in my experience. For what it's worth Plan 9 has excellent concurrent programming facilities.
>Or that wouldn’t use GPUs?
GPUs these days only work on systems in large part due to graces of the vendors themselves. Would nvidia or AMD be usable (enough) on Linux if there was not dedicated people from those companies working on drivers?