Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory Safety

memorysafety.org

Post reply on HN
Print view View original

121–130 of 157 posts

Re: Memory Safety

#121

Post by creatonez »

fweimer wrote: post #109

I find it strange that this web site completely ignores the Java ecosystem, which offers memory-safe implementations for most of the protocols and services listed.

Java does fine on memory safety, but does not do great on null safety (and overall invariant protection / "make invalid states unrepresentative" ethos), has difficult to harden concurrency primitives, and won't be adopted in many scenarios due to runtime cost and performance pitfalls. Future Valhalla work fixes some of these issues, but leaves many things spiky.

Re: Memory Safety

#122

Post by lenkite »

lomase wrote: post #21
K0nserv wrote: post #11

Earlier quoted context omitted.

Changes to multi-word pointers can cause UB due to race conditions in Go because only changes at the word level are atomic. See: https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...

Does Rust not have race conditions?

Rust has double free concurrency bugs

https://materialize.com/blog/rust-concurrency-bug-unbounded-...

Lockbud: project detailing memory, concurrency bugs and panics for Rust. https://github.com/BurtonQin/lockbud

USENIX paper on model checking for Rust OS kernels uncovered 20 concurrency bugs across 12 modules in projects like Redox OS and Tock, including data races, deadlocks, and livelocks

https://www.usenix.org/system/files/atc25-tang.pdf

Re: Memory Safety

#123

Post by fweimer »

creatonez wrote: post #121
fweimer wrote: post #109

I find it strange that this web site completely ignores the Java ecosystem, which offers memory-safe implementations for most of the protocols and services listed.

Java does fine on memory safety, but does not do great on null safety (and overall invariant protection / "make invalid states unrepresentative" ethos), has difficult to harden concurrency primitives, and won't be adopted in many scenarios due to runtime cost and performance pitfalls. Future Valhalla work fixes some of these issues, but leaves many things spiky.

I dislike Java's abstraction-through-indirection approach, which is related to the non-representable invalid states you mention. But I think it's more of a matter of taste.

Somewhat controversially, I think Java is actually doing fine on null safety: it uses the same approach for it as it does for array index safety. The latter is a problem for any language with arrays without dependent types: out-of-bounds accesses (if detectable at all) result in exceptions (often named differently because exceptions are controversial).

Java's advantage here is that it doesn't pretend that it doesn't have exceptions. I think it's quite rare to take down a service because handling a specific request resulted in an exception. Catching exceptions, logging them, and continuing seems to be rather common. It's not like Rust and Go, where unexpected panics in libraries are often treated as security vulnerabilities because panics are expected to take down entire services, instead of just stopping processing of the current request.

Re: Memory Safety

#124

Post by checker659 »

fweimer wrote: post #111
checker659 wrote: post #97

I never understood why software has to pay for the lack of memory safety primitives in the hardware.

People have tried, and so far, achieving safety through trusted compilers and (fairly complicated) run-time support has been much more efficient. A small team could probably design a RISC-V CPU with extensions for hardware-assisted bounds checking and garbage collection, but any real CPU that they can built would likely have performance levels that are typical for research-oriented RISC-V CPUs. Doing the same thing i…

See that's the problem. Unless this is government mandated, no sane vendor is going to pay for the performance penalty.

> Doing the same thing in software on a contemporary commercially established CPU is going to be much, much faster.

In what sense? Do you know if there's been proper research done in this area? Surely implementing the bounds checking / permissions would be faster in hardware.

Re: Memory Safety

#125

Post by fweimer »

bheadmaster wrote: post #115
fweimer wrote: post #109

I find it strange that this web site completely ignores the Java ecosystem, which offers memory-safe implementations for most of the protocols and services listed.

Can I use the Java implementations in another language without significant headache?

It's possible in practice (at least more so than with Go), but it's highly unusual. Back when free Java became a thing, I used it at first to obtain a memory-safe TLS implementation. It worked out well, I think, but there is a strong tendency for the JVM to become the trunk of your application that holds everything together.

Re: Memory Safety

#126

Post by creatonez »

fweimer wrote: post #123
creatonez wrote: post #121

Earlier quoted context omitted.

Java does fine on memory safety, but does not do great on null safety (and overall invariant protection / "make invalid states unrepresentative" ethos), has difficult to harden concurrency primitives, and won't be adopted in many scenarios due to runtime cost and performance pitfalls. Future Valhalla work fixes some of these issues, but leaves many things spiky.

I dislike Java's abstraction-through-indirection approach, which is related to the non-representable invalid states you mention. But I think it's more of a matter of taste. Somewhat controversially, I think Java is actually doing fine on null safety: it uses the same approach for it as it does for array index safety. The latter is a problem for any language with arrays without dependent types: out-of-bounds accesses…

I'm not talking about null safety in the sense of null pointers. Null pointers and out of bound pointers are still in the realm of memory safety, which of course Java has solved for the most part.

Proper null safety (sometimes called void safety) is to actually systematically eliminate null values, to force in the type system a path of either handling or explicitly crashing. This is what many newer expressive multi-paradigm languages have been able to achieve (and something functional programming languages have been doing for ages), but remains out of reach for Java. Java does throw an exception on errant null value access, but allows the programmer to forget to handle it by making it a `RuntimeException`, and by the time you might try to handle it, you've lost all of the semantics of what went wrong - what value was actually missing and what a missing value truly means in the domain.

> Catching exceptions, logging them, and continuing seems to be rather common. It's not like Rust and Go, where unexpected panics in libraries are often treated as security vulnerabilities because panics are expected to take down entire services, instead of just stopping processing of the current request.

Comparing exceptions to panics is a category error. Rust for example has great facilities for bubbling up errors as values. Part of why you want to avoid panicking so much is that you don't need to do it, because it is just as easy to create structured errors that can be ignored by the consumer if needed. Java exceptions should be compared to how errors are actually handled in Rust code, it turns out they end up being fairly similar in what you get out of it.

Re: Memory Safety

#127

Post by pizlonator »

vlovich123 wrote: post #91
pizlonator wrote: post #54

Earlier quoted context omitted.

> By definition, C and C++ are memory safe as long as you follow the rules. This statement doesn't make sense to me. Memory safety is a property of language implementations, which is all about what happens when the programmer does not follow the rules. > The problem is that the rules cannot be automatically checked and in practice are the source of unenumerable issues from straight up bugs to subtle standards violati…

First, let me say that I really respect the work you’re doing in fil-c. Nothing I say is intended as a knock and you’re doing fantastic engineering work moving the field forward and I hope you find success. That’s good to know about nasal demons. Are you saying you somehow inhibit the optimizer from injecting a security vulnerability due to UB ala https://www.cve.org/CVERecord?id=CVE-2009-1897 ? I’m kinda curious how…

> Are you saying you somehow inhibit the optimizer from injecting a security vulnerability due to UB ala https://www.cve.org/CVERecord?id=CVE-2009-1897 ? I’m kinda curious how you trick LLVM into not optimizing through UB since it’s UB model is so tuned to the C/C++ standard.

Yes that is inhibited. There’s no trick. LLVM (and other compilers) choose to do those stupid things by policy, and the policy can be turned off. It’s not even hard to do it.

> Fil-C is more an independent dialect of C/C++ (eg you do have to patch some existing software)

Fil-C is not a dialect. The patches are similar to what you’d have to do if you were porting a C program to a new CPU architecture or a different compiler.

> By this argument no language is memory safe because every language has bugs that can result in memory safety issues.

You rebutted this argument for me:

> any safety issues within the language spec or implementation are a bug to be fixed

Exactly this. A memory safe language implementation treats outstanding memory safety issues as a bug to be fixed.

This is what makes almost all JS implementations, and Fil-C, memory safe.

Re: Memory Safety

#128

Post by fweimer »

checker659 wrote: post #124
fweimer wrote: post #111

Earlier quoted context omitted.

People have tried, and so far, achieving safety through trusted compilers and (fairly complicated) run-time support has been much more efficient. A small team could probably design a RISC-V CPU with extensions for hardware-assisted bounds checking and garbage collection, but any real CPU that they can built would likely have performance levels that are typical for research-oriented RISC-V CPUs. Doing the same thing i…

See that's the problem. Unless this is government mandated, no sane vendor is going to pay for the performance penalty. > Doing the same thing in software on a contemporary commercially established CPU is going to be much, much faster. In what sense? Do you know if there's been proper research done in this area? Surely implementing the bounds checking / permissions would be faster in hardware.

I'm worried that if memory tagging becomes mandatory, it sucks the air out of the room for solutions that might have a more long-lasting impact. Keep in mind that memory tagging is just heuristics beyond very specific bug scenarios (linear buffer overflows are the prime example). The whole thing does not seem fundamentally resistant to future adaptions of exploitation techniques. (Although oddly enough, I have been working on memory tagging lately.)

Regarding performant implementations of capability architectures, Fil-C running on modern CPUs is eventually going to overtake Arm's Morello reference board because it doesn't look like there's going to be a successor to the board. Morello was based on Arm's Neoverse-N1 core and produced using TSMC's N7 process. It was a research project, but it's really an outlier because such projects hardly ever have access to these kinds of resources (both CPU IP and tape-out on a previous-generation process). It seems all other implementations of CHERI are FPGA-based.

Re: Memory Safety

#129

Post by pizlonator »

fweimer wrote: post #103
pizlonator wrote: post #65

Earlier quoted context omitted.

Canonical example: union { char* p; long i; }; Then say that the attacker can write arbitrary integers into `i` and then trigger dereferences on `p`.

The standard does not assign meaning to this sequence of execution, so an implementation can detect this and abort. This is not just hypothetical: existing implementations with pointer capabilities (Fil-C, CHERI targets, possibly even compilers for IBM i) already do this. Of course, such C implementations are not widely used. The union example is not particularly problematic in this regard. Much more challenging is p…

> The standard does not assign meaning to this sequence of execution, so an implementation can detect this and abort.

Real C programs use these kinds of unions and real C compilers ascribe bitcast semantics to this union. LLVM has a lot of heavy machinery to make sure that the programmer gets exactly what then expected here.

The spec is brain damage. You should ignore it if you want to be able to reason about C.

> This is not just hypothetical: existing implementations with pointer capabilities (Fil-C, CHERI targets, possibly even compilers for IBM i) already do this

Fil-C does not abort when you use this union. You get memory safe semantics:

- you can use `i` to change the pointer’s intval. But the capability can’t be changed that way. So if you make a mistake you’ll end up with an OOB pointer.

- you can use `i` to read the pointer’s current intval just as if you had done an ptrtoint cast.

I think CHERI also does not abort on the union itself. I think storing to `i` removes the capability bit so `p` crashes on deref.

> The union example is not particularly problematic in this regard. Much more challenging is pointer arithmetic through uintptr_t because it's quite common.

The union problem is one of the reasons why C is not memory safe, because C compilers give unions the expected structured assembly semantics, not whatever nonsense is in the spec.

Re: Memory Safety

#130

Post by fweimer »

creatonez wrote: post #126
fweimer wrote: post #123

Earlier quoted context omitted.

I dislike Java's abstraction-through-indirection approach, which is related to the non-representable invalid states you mention. But I think it's more of a matter of taste. Somewhat controversially, I think Java is actually doing fine on null safety: it uses the same approach for it as it does for array index safety. The latter is a problem for any language with arrays without dependent types: out-of-bounds accesses…

I'm not talking about null safety in the sense of null pointers. Null pointers and out of bound pointers are still in the realm of memory safety, which of course Java has solved for the most part. Proper null safety (sometimes called void safety) is to actually systematically eliminate null values, to force in the type system a path of either handling or explicitly crashing. This is what many newer expressive multi-p…

I understood what you meant. I just disagree about priorities. Conceptually, every array access (absent dependent types) can produce a null value because the index might be out of bounds. Languages that eliminate null values in other areas typically fail to deal with the array indexing issue at the type level, which seems at least as prevalent in real-world code as null pointer deferences, if not more so.

Regarding the category error, on many platforms, Rust panics use the same underlying implementation mechanism as C++ exceptions. In general, Rust library code is expected to be panic-safe. Some well-known Rust tools use panics for control flow (in the same way one would abuse exceptions). The standard test framework depends on recoverable panics, if I recall correctly. The Rust language gives exceptions a different name and does not provide convenient syntax for handling them, but it still has to deal with the baggage associated with them, and so do Rust library authors who do not want to place restrictions on how their code is reused. It's not necessarily a bad approach, to be clear: avoiding out-of-bounds indexing errors completely is hard.

Post reply on HN