Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safety is necessary, not sufficient

steveklabnik.com

Post reply on HN
Print view View original

121–130 of 162 posts

Re: Memory safety is necessary, not sufficient

#121

Post by jholman »

fulafel wrote: post #107
dgreensp wrote: post #19

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

"Worse is better" is much older then "undefined behaviour". Undefined behaviour was invented for C standardization, when C was already mature and had been out of the Unix childhood home for a long time. For example, many UB semantics in the standard come from allowing for C ports for strange non-unix hardware, and making bold decisions when developments in compiler optimization state of the art ran into underspecifie…

"Worse Is Better", as a description of a set of engineering values embodied by C, is from 1989.

I'm taking your word for it that UB was invented for C standardization (I have no knowledge of that history, and your claim seems plausible), and I'm going to say that probably they didn't invent it in the last year of the seven-year standardization effort, so probably UB is older than Worse Is Better.

Re: Memory safety is necessary, not sufficient

#122

Post by mattpallissard »

dgreensp wrote: post #19

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

> It was a very different world, and just being able to write in a higher-level language than assembly on a particular computer was a big deal.

It was different then, but not by too much. Our tooling, languages, etc, has improved yes, but the underlying hardware still exists and occasionally the general solution provided by our improved tooling is sub optimal for our use case.

I actually _like_ where C has wound up. I have way better static analysis tooling at my disposal than I had 15 years ago and I can use a "memory safe" language like python or OCaml and say "hey, I know better here", think carefully, and push the memory safe compiler out of my way.

Re: Memory safety is necessary, not sufficient

#123

Post by trimistermota »

wavemode wrote: post #60
dgreensp wrote: post #19

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

Software was significantly simpler in those days. I don't find it strange that they took a simplified view of software engineering. Specifically because those exact same simplified views still exist today - talk to people who've never worked on large complex systems, and you will usually encounter similar "anti-perfectionism". People adopt simplified engineering practices when working on simple software by themselves…

One of the reasons C became popular because the compiler was actually reasonable to use on the hardware of the time.

Re: Memory safety is necessary, not sufficient

#124

Post by omnicognate »

SnowflakeOnIce wrote: post #79
tptacek wrote: post #35

I'm not sure I understand what this piece is trying to say about Python memory safety. Conventionally, in software security, Python is considered a memory-safe language. The piece makes the case that Python isn't memory safe when you FFI into a C library. But neither is Rust, nor is it when you use `unsafe`. What matters in both case is how little unsafe code you end up writing. Memory safety is a software security c…

> Conventionally, in software security, Python is considered a memory-safe language. The piece makes the case that Python isn't memory safe when you FFI into a C library. Interesting and largely unknown trivia: it's possible to invoke memory errors in the underlying C interpreter from pure Python code — no libraries and no imports needed! One way of doing this is by creating new `code` objects with crafted bytecode.…

You also don't need to FFI into some buggy C library to violate memory safety with ctypes. It's trivial to produce a segfault with it without using anything but ctypes itself, which is part of the standard library. I doubt I'd have much trouble finding other ways to make a segfault with pure python and the standard library (struct springs to mind).

CPython really isn't very safe at all. Its focus has always been on being a convenient, dynamic scripting language with minimal-fuss access to native code. It has never been hard to violate its internal assumptions and it probably never will be.

And I'm pretty comfortable with that, FWIW.

Re: Memory safety is necessary, not sufficient

#125

Post by trimistermota »

mattpallissard wrote: post #122
dgreensp wrote: post #19

I think it’s worth emphasizing that the C spec’s love of undefined behavior—if you do X by accident, anything can happen—and the apparently massive amount of memory-unsafe software that has been written that will just allocate 16 bytes on the stack and then read from a file descriptor until it encounters a null byte… are examples of things that aren’t considered remotely sane or reasonable to a modern programmer or l…

> It was a very different world, and just being able to write in a higher-level language than assembly on a particular computer was a big deal. It was different then, but not by too much. Our tooling, languages, etc, has improved yes, but the underlying hardware still exists and occasionally the general solution provided by our improved tooling is sub optimal for our use case. I actually _like_ where C has wound up.…

One of the reasons C became popular because the compiler was actually reasonable to use on the hardware of the time.

Re: Memory safety is necessary, not sufficient

#126

Post by justincormack »

amelius wrote: post #22

Our programs are growing so big by having so many (indirect) dependencies that we need a way to sandbox the libraries that we include from our main programs. This is the type of safety that I'm looking for, really.

Some people are looking at Wasm as a cross language way of doing this. Some work in language specific ways is also going on. Definitely an interesting area.

Re: Memory safety is necessary, not sufficient

#127

Post by trissylegs »

nikic wrote: post #118
moody__ wrote: post #74

Earlier quoted context omitted.

>it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. This is true and when your compiler actually abides by these values it is shocking how many issues just go away. The problem is that gcc and clang are nowhere near a simple implementation, asking the question of how exactly gcc or clang arrived at some given assembly for some giv…

> People can (and do) point at the C spec for fault of this and it is true that if the C spec was more strict then these compilers would not have the free pass to do these crazy miscompilations. However there is nothing stopping these compilers from just not doing that, there is nothing stopping them from just defining their own sane behavior for what the C spec defines as undefined behavior. It is no longer the case…

For example Linux uses -fno-delete-null-pointer-checks

Re: Memory safety is necessary, not sufficient

#128

Post by quelsolaar »

kaba0 wrote: post #97
quelsolaar wrote: post #33

Earlier quoted context omitted.

C gives you a level of control and responsibility not found in other languages. That's a choice, not something that is inherently worse. It may be worse for what you are doing. Most people don't value the level of control that C gives you and would rather chose another language and that is fine. But having a language available with this level of control is valuable, even if few people chose to use it. Most UB in the…

C is not a particularly low-level. It has no (standard) way to control vectorization, stack usage, calling conventions, etc. It just had an insane about of money spent on making its compilers optimize better.

You are right, but no language has tried to claim the space of giving the user more control, so C remains the lowest level language we have that is portable. I think there are lots of opportunities in this space, but i don't know of anyone working on it.

Re: Memory safety is necessary, not sufficient

#129

Post by Buttons840 »

nikic wrote: post #118
moody__ wrote: post #74

Earlier quoted context omitted.

>it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. This is true and when your compiler actually abides by these values it is shocking how many issues just go away. The problem is that gcc and clang are nowhere near a simple implementation, asking the question of how exactly gcc or clang arrived at some given assembly for some giv…

> People can (and do) point at the C spec for fault of this and it is true that if the C spec was more strict then these compilers would not have the free pass to do these crazy miscompilations. However there is nothing stopping these compilers from just not doing that, there is nothing stopping them from just defining their own sane behavior for what the C spec defines as undefined behavior. It is no longer the case…

> The key thing to understand is that if you use these flags, you opt-in to a non-standard C dialect.

Isn't all C with undefined behavior non-standard. Or is there a standard for undefined behavior (obviously not, I would think)?

I don't understand why those flags wouldn't be turned on by default. Or do they affect more than just undefined behavior?

Re: Memory safety is necessary, not sufficient

#130

Post by petergeoghegan »

nikic wrote: post #118
moody__ wrote: post #74

Earlier quoted context omitted.

>it is more important that the implementation—or the design of the implementation—of a piece of software be simple than that it be correct. This is true and when your compiler actually abides by these values it is shocking how many issues just go away. The problem is that gcc and clang are nowhere near a simple implementation, asking the question of how exactly gcc or clang arrived at some given assembly for some giv…

> People can (and do) point at the C spec for fault of this and it is true that if the C spec was more strict then these compilers would not have the free pass to do these crazy miscompilations. However there is nothing stopping these compilers from just not doing that, there is nothing stopping them from just defining their own sane behavior for what the C spec defines as undefined behavior. It is no longer the case…

> Your code can no longer be compiled by a compiler that does not support these special C dialects.

What compiler might that be?

It's true that MSVC doesn't have an equivalent of -fno-strict-aliasing, but that's because it just doesn't apply optimizations that assume strict aliasing in the first place. Admittedly the picture around signed overflow is more complicated, but it's essentially the same story.

> Using these C dialects is a perfectly sensible thing to do, as long as you understand that this is what you are doing.

I suppose that they technically are C dialects, but it seem more than a bit absurd to put it like that -- at least to me. By that standard the Linux kernel isn't written in C. And Firefox isn't written in C++. Postgres also wouldn't count as according-to-hoyle C under your definition. Even though Postgres compiles when -fwrapv and -fno-strict-aliasing are removed, and still passes all tests.

The implication of what you're saying seems to be that all of these open source projects each independently decided to "go there own way". I find it far more natural to explain the situation as one of GCC diverging when it decided to make -fstrict-aliasing the default around 15 years ago.

Post reply on HN