Live data from Hacker News

Memory safety is necessary, not sufficient

steveklabnik.com

111–120 of 162 posts

Re: Memory safety is necessary, not sufficient

#111

Nobody has been able explain to me what would be lost if we defined data races to yield one of the values that had been written to the memory in the past, instead of being undefined. It is not as if any optimizer can see that you are racing and delete the code path that has it.

Let me _try_ to explain it. One reason is that non-atomic writes can be torn. So if you have a value like 0x00000000 and over write it with 0xFFFFFFFF, some hardware may do it as two separate writes. This means that another thread can read the data when its half way written and get 0xFFFF0000. I'm using a 32bit value here to illustrate In reality modern hardware is unlikely to tear it, but in other cases it may. Anot…

“Fun” stuff: Rust also doesn’t prevent this for more complicated objects. The ‘Sync’ trait will get applied to any struct that has only ‘Sync’ constituents, but that means that such an object can be observable in inconsistent states (e.g. a strange Date object where year, month and day are all atomic numbers can be 2023-02-31).

Re: Memory safety is necessary, not sufficient

#112

Earlier quoted context omitted.

Just like in every other kind of engineering, for software capability pulls the requirements. The only reason everybody is complaining about lack of memory safety now is because there are alternatives for every use case. Before Rust existed, it was seen as an inevitable issue that one must work with, not as a problem to solve.

>Before Rust existed, it was seen as an inevitable issue that one must work with, not as a problem to solve. There were many memory safe languages before Rust, Java and C# evangelized this a lot, though at that time the accent was on memory leaks not on safety. Rust offers an alternative for low level programming and might be faster in soem scenarios then managed languages.

> might be faster in soem scenarios

in which scenarios would it not be faster?

Re: Memory safety is necessary, not sufficient

#113

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

> If you read the essay, it says “worse is better” means (paraphrasing) it’s more important that C compilers be easy to implement than easy to use. Also, it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. It is more important that it be simple than that it be consistent. It is more important that it be simple than that it be “comp…

Zero cost abstractions & backward compatibility over implementation complexity, user friendliness or safety.

Re: Memory safety is necessary, not sufficient

#114
post #76

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

Undefined behavior is critical for performance. Without undefined behavior, C compilers would not be able to optimize at all. You'd be running everything at -O0 or worse.

Then fence it in and have it be as little of it as possible and as obvious as possible when it can happen.

Re: Memory safety is necessary, not sufficient

#115
post #92

Earlier quoted context omitted.

Even if that's true (which is not given, as others have said), it would be a worthy trade to make. Software needs to do what it's meant to do first and foremost. Speed doesn't mean shit if you can't trust that the software actually works.

But that is simply not true, and can be proved by looking at the world we live in. C became the dominant language precisely due to hardware constraints, and the ability to extract every last drop from limited hardware was back in the day more important than software working perfectly always. If this wasn't the case, other safer alternatives would have been preferred. Unless in very specific domains, hardware advances…

C became dominat, because UNIX was a free beer OS, with source tapes and a book to come along for the ride (Lion's commentary).

Had UNIX been as expensive as VMS, or System/370, with a commercial license, no university would have cared to port UNIX, and focus on the systems language used to develop it (post-UNIX V5).

As for its performance myth, regarding 1980's C compilers.

"Oh, it was quite a while ago. I kind of stopped when C came out. That was a big blow. We were making so much good progress on optimizations and transformations. We were getting rid of just one nice problem after another. When C came out, at one of the SIGPLAN compiler conferences, there was a debate between Steve Johnson from Bell Labs, who was supporting C, and one of our people, Bill Harrison, who was working on a project that I had at that time supporting automatic optimization...The nubbin of the debate was Steve's defense of not having to build optimizers anymore because the programmer would take care of it. That it was really a programmer's issue.... Seibel: Do you think C is a reasonable language if they had restricted its use to operating-system kernels? Allen: Oh, yeah. That would have been fine. And, in fact, you need to have something like that, something where experts can really fine-tune without big bottlenecks because those are key problems to solve. By 1960, we had a long list of amazing languages: Lisp, APL, Fortran, COBOL, Algol 60. These are higher-level than C. We have seriously regressed, since C developed. C has destroyed our ability to advance the state of the art in automatic optimization, automatic parallelization, automatic mapping of a high-level language to the machine. This is one of the reasons compilers are ... basically not taught much anymore in the colleges and universities."

-- Fran Allen interview, Excerpted from: Peter Seibel. Coders at Work: Reflections on the Craft of Programming

Re: Memory safety is necessary, not sufficient

#116

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

C gives you a level of control and responsibility not found in other languages. That's a choice, not something that is inherently worse. It may be worse for what you are doing. Most people don't value the level of control that C gives you and would rather chose another language and that is fine. But having a language available with this level of control is valuable, even if few people chose to use it. Most UB in the…

Whatever people it is C for low level coding, in reality are compiler specific language extensions, not available on ISO C.

All languages can have such specific extensions and many do.

Re: Memory safety is necessary, not sufficient

#117
post #98
post #42

Earlier quoted context omitted.

Citation welcome to those case studies, because I've not seen them. It's on the advocates of rust to establish that it makes things better because it absolutely isn't unambiguous. We really seem to be in the stone age in terms of what practices lead to higher quality software. We still have people who chant "goto harmful" against one simply forward jumps to on-error-cleanup code, yet still litter their C++ and java w…

How are exceptions less safe or less clear? Also, crashing fast is the best way to deal with unforeseen events one can’t recover from. Your anecdotal experience with regards to firefox and rust just shows that they put more assertions into the code (good!), which makes it easier to notice and later fix bugs, as rust makes it mandatory to handle error cases to some degree. This is also a plus for rust.

If you can't recover, perhaps. But when it causes more unrecoverable states (e.g. because handling the case requires a massive refactor to satisfy the borrow checker, so you just panic...) it's another issue.

Even ignoring that, not everything is a security sensitive internet application. In plenty of cases trundling on with a corrupt state in production is superior, particularly since many corrupt states are benign (particularly randomly occurring ones, rather than attacker produced ones). E.g. is it better for the software driving your Christmas lights to potentially glitch for a moment and display the wrong colors or to just shut down (/get stuck, depending on the design)?

> as rust makes it mandatory to handle error cases to some degree

"handle" often just means panic, and in cases where security isn't a concern and there isn't any persistent data to corrupt crashing may already be the worst thing that could happen. So you might have that the C-written program would have correctly handled the case but due to rust's effort front loading it just doesn't get handled. ... but even if the C-written program handled it wrong it would be worse than the rust code that panicked (for some applications) and quite possibly better.

I've experienced rust sometimes normalizes intentionally writing code that is effectively if()elif()elif()else{abort();} as a result of the a culture of panic being 'safe' and making other choices require more upfront effort.

For Firefox which is generally security critical I fully agree that panicing is better than being corrupt. But we'd be assuming facts not in evidence if we assume that the panics would actually be bad states if that code were written in C++, it's possible that some of them would have been correctly handled but for the extra effort rust required up front. In firefox the tradeoff for more defects in exchange for being safer is still probably a good one. But one can't be in denial of this possibility if one is to minimize the cost of rust, apply it to where it's most applicable, etc.

> How are exceptions less safe or less clear?

Potentially! It depends on the culture of their use. They have a highly non-local effect, so some particular weird exception happens in library of library code of a sort that is completely incomprehensible to the code 20 steps back up the stack. And not just incomprehensible, but unknowable since the called code could be changed later and add exceptions that the caller couldn't have known of. The control flow diverts away for an invisible and potentially unknowable reason. For that reason many places have varrious rules about exceptions, including sometimes prohibiting them entirely.

Of course it can be used well and safely too with care.

But so can "goto fail; ... fail:...". And code that could locally handle an error case by unwidindign itself and e.g. giving an empty result probably ought to do so rather than bubble up any one of thirteen different exception states that may be differently mishandled. (or at least the better choice between the approaches isn't something that can be correctly answered by a maxim or anything less than case specific judgement)

Re: Memory safety is necessary, not sufficient

#118
post #74

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

>it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. This is true and when your compiler actually abides by these values it is shocking how many issues just go away. The problem is that gcc and clang are nowhere near a simple implementation, asking the question of how exactly gcc or clang arrived at some given assembly for some giv…

> People can (and do) point at the C spec for fault of this and it is true that if the C spec was more strict then these compilers would not have the free pass to do these crazy miscompilations. However there is nothing stopping these compilers from just not doing that, there is nothing stopping them from just defining their own sane behavior for what the C spec defines as undefined behavior. It is no longer the case where we have a dozen or so C compilers that people need to target with their programs, the spec is not the bottom line anymore.

Some compilers (like gcc and clang) do give you the option to make most undefined behaviors well-defined, using flags like -fwrapv, -fno-delete-null-pointer-checks, -fno-strict-aliasing, etc.

The key thing to understand is that if you use these flags, you opt-in to a non-standard C dialect. Going forward, you are no longer writing C code. Your code is now non-portable. Your code can no longer be compiled by a compiler that does not support these special C dialects.

Using these C dialects is a perfectly sensible thing to do, as long as you understand that this is what you are doing.

Re: Memory safety is necessary, not sufficient

#119
post #104
post #9

Earlier quoted context omitted.

I saw the authors credentials and I do respect them a lot. But to be fair, I'm sure the person who wrote the Go manual could have written the same blog post with the same outlook for the future only with Go in place of Rust. I'm trying to broaden the scope of conversation to a more holistic one, rather than just "this is our chance to take over the world!" Like my Gotek USB emulator reference. The device costs $50, a…

Go has absolutely zero interesting properties here. It has a shitty, inexpressive type system, dangerous concurrency, brain-dead error handling. It is a managed language, which provided memory-safety for many many decades now.

> which provided memory-safety for many many decades now.

That is a plus, not a minus.

Re: Memory safety is necessary, not sufficient

#120
post #42

Earlier quoted context omitted.

Citation welcome to those case studies, because I've not seen them. It's on the advocates of rust to establish that it makes things better because it absolutely isn't unambiguous. We really seem to be in the stone age in terms of what practices lead to higher quality software. We still have people who chant "goto harmful" against one simply forward jumps to on-error-cleanup code, yet still litter their C++ and java w…

> the vast majority of firefox crashes I experience now are rust panics Have you considered that this might be precisely because Rust is catching and stopping something bad with a safe panic? And that the same programmer making the same mistake in C/C++ would have resulted in silent data corruption, a security vulnerability, and possibly a portal into the space between dimensions from which optimising compilers will…

> Have you considered that this might be precisely because Rust is catching and stopping something bad with a safe panic

Absolutely! But I counter: How do you know that all of them are? Or that even many of them are?

The developers that work on it may well know, but it isn't something we can axiomatically assume. It is not a true statement that any rust panic would have been an error in C(++) code written by the same (competent in both) developer.

Increased error visibility can be either more errors or more error sensitivity. It's worth being mindful of this because otherwise we may adopt programming practices that increase the overall defect rate, and think we're making things better.

Perhaps it's easier to see with asserts. If your defect rate is 1 defect per 1000 lines of code, and you go add 1000 asserts you'll add a bug (actually I think you'll add 10: the boundary conditions that asserts test are far more likely than average code to be implemented wrong). Like any other code they could also start off correct but then become desynced with the rest of the code. If you go and ship those asserts in production and the effect of asserting is important in your application, then unless the asserts prevent more bugs than they created you made the program worse off. Even if the asserts don't cause bugs they may make the program harder to maintain or extend. Therefore in any codebase there must be some optimal level of asserts, more or less and the program is worse off.

Is the implicit assertion of rust's behavior optimal? Probably not for all codes, because different codes call for different tradeoffs. Or even for every code there might be language changes that could make things better. But we'll never find out this stuff if we can't admit that there is the potential for uncertainty or improvement and confuse memory safety for program correctness. It's one of the more important aspects of correctness but the world is full of serious bugs in software written in inherently memory safe languages.

And after all, a program consisting of nothing but exit(1) (or perhaps while(;;){}) is the most perfectly memory safe program possible. :P If memory safety were the only goal programming would be much simpler.

> and possibly a portal into the space between dimensions from which optimising compilers will happily allow the Others into our world

As mentioned up thread, rust also can create portals into the hell dimension if there is unsafe code (including in the standard library) that fails to obey all the not-very-simple contract requirements. Also the number of times errant C code has actually opened a portal in to the hell dimension thereby destroying the universe is greatly overstated, that's only happened twice at most, usually undefined behavior is just a crash or an exploitable misbehavior that lets some haxor steal your data and those can also happen from pure logic errors (or a panic, in the case of a crash). I do fully agree that narrowing the potential for UB misbehavior is very important and rust is an important advance there. But Rust doesn't just do that. It also has its own costs and quirks.

Post reply on HN