Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is no memory safety without thread safety

ralfj.de

Post reply on HN
Print view View original

281–290 of 517 posts

Re: There is no memory safety without thread safety

#281

Post by int_19h »

wpollock wrote: post #247
ferreiratb wrote: post #229

Earlier quoted context omitted.

Can you elaborate on that?

A Java program can share mutable state between threads without synchronization, and it will compile and run. In Rust, such a program will not compile.

Yes, but even so you will never see e.g. an invalid pointer value as the result of a torn memory write. Basically, no matter what you do with threads in Java, it will not segfault.

TFA's point is that (safe) Rust is also like that, but achieves it by restricting all cases where a torn write could be observed through its type system instead of VM's memory model.

Re: There is no memory safety without thread safety

#282

Post by int_19h »

shadowgovt wrote: post #74
chadaustin wrote: post #62

Every time this conversation comes up, I'm reminded of my team at Dropbox, where it was a rite of passage for new engineers to introduce a segfault in our Go server by not synchronizing writes to a data structure. Swift has (had?) the same issue and I had to write a program to illustrate that Swift is (was?) perfectly happy to segfault under shared access to data structures. Go has never been memory-safe (in the Rust…

Mostly because it was a remarkable improvement over what came before (and what came before was hilariously fragile).

It was certainly not a remarkable improvement in the sense of being memory safe even in the face of race conditions. As the article points out, Java and C# both managed to do that, and both predate Go.

Re: There is no memory safety without thread safety

#283

Post by int_19h »

tomp wrote: post #161
recursivecaveat wrote: post #119

Earlier quoted context omitted.

The situation with numbers in basically every widely used programming language is kind of an indictment of our industry. Silent overflow for incorrect results, no convenient facilities for units, lossy casts everywhere. It's one of those things where standing in 1975 you'd think surely we'll spend some of the next 40 years of performance gains to give ourselves nice, correct numbers to work with, but we never did.

"nice, correct numbers" end somewhere between 1/3 and sqrt(2) so in reality, it's just "pick your own poison" to various degrees...

Even so, we don't need to pick the more potent poison. And most certainly not when we had decades of awareness about better alternatives.

There's really no excuse for a modern PL to not have, at the very least, overflow detection by default.

Re: There is no memory safety without thread safety

#284

Post by tsimionescu »

potato-peeler wrote: post #136
chadaustin wrote: post #62

Every time this conversation comes up, I'm reminded of my team at Dropbox, where it was a rite of passage for new engineers to introduce a segfault in our Go server by not synchronizing writes to a data structure. Swift has (had?) the same issue and I had to write a program to illustrate that Swift is (was?) perfectly happy to segfault under shared access to data structures. Go has never been memory-safe (in the Rust…

I am curious. Generally basic structures like map are not thread safe and care has to be taken while modifying it. This is pretty well documented in go spec. In your case in dropbox, what was essentially going on?

I think the surprise here is that failing to synchronize writes leads to a SEGFAULT, not a panic or an error. This is the point GP was making, that Go is not fully memory safe in the presence of unsynchronized concurrent writes. By contrast, in Java or C#, unsynchronized writes will either throw an exception (if you're lucky and they get detected) or let the program continue with some unexpected values (possibly ones that violate some invariants). Getting a SEGFAULT can only happen if you're explicitly using native code, raw memory access APIs, or found a bug in the runtime.

Re: There is no memory safety without thread safety

#285

Post by SkiFire13 »

nine_k wrote: post #120
crawshaw wrote: post #103

Memory safety is a big deal because many of the CVEs against C programs are memory safety bugs. Thread safety is not a major source of CVEs against Go programs. It’s a nice theoretical argument but doesn’t hold up in practice.

A typical memory safety issue in a C program is likely to generate an RCE. A thread-safety issue that leads to a segfault can likely only lead to a DoS attack, unpleasant but much less dangerous. A race condition can theoretically lead to more powerful attacks, but triggering it should be much harder.

A thread-safety issue does not always lead to a segfault. Here it did because the address written was 42, but if you somehow manage to obtain the address of some valid value then you could read from that instead, and not cause an immediate segfault.

I agree with the sentiment that data races are generally harder to exploit, but it _is possible_ to do.

Re: There is no memory safety without thread safety

#286

Post by NitpickLawyer »

tptacek wrote: post #272
stickfigure wrote: post #270

Earlier quoted context omitted.

I have recently come to the conclusion that everything I ever thought was "contrived" is currently standard practice in some large presently existing organization.

Take that to the Apple bounty program with your crasher bug and tell them they should pay out as if you'd confirmed RCE, see how it goes. This is an engineering question; it's not about vibes. It's not even always the case that corrupted data structures (or even pointers) in C code are exploitable. You need attacker control of data and where it goes in memory. It's far less often the case in Python or Go --- in fact,…

Dunno about Apple, but goog sometimes pays out bugs that are "theoretical" in the way you describe. That is, you show that there's a bug somewhere, but you can't "reach" it from user data. They'll pay less than a PoC, obviously, but will pay. YMMV, etc.

Re: There is no memory safety without thread safety

#287

Post by uecker »

okanat wrote: post #243
Waterluvian wrote: post #238

Earlier quoted context omitted.

This is way outside my domain but isn’t the answer: yes, if the code is formally proven safe? Doesn’t NASA have an incredibly strict, specific set of standards for writing safety critical C that helps with writing programs that can be formalized?

There are safety recommendations / best practice standards like CERT. None of them will prevent you from making intentional looking but logically unsound memory unsafe operations with C and C++. The code can be very indistinguishable from safe code. The things that C and C++ allow you to do basically makes code written in those languages impossible to fully formally prove. Although there are subsets, the basic intege…

There is also a lot of formally verified in C.

Re: There is no memory safety without thread safety

#288

Post by tsimionescu »

pjmlp wrote: post #102
9rx wrote: post #95

Earlier quoted context omitted.

On the flip side, what would be the point? There are already a million other languages that have everything and the kitchen sink. Not going down the same road is the only reason it didn't end up on the pile of obscure languages nobody uses.

The only reason it didn't end on pile of obscure languages nobody uses, it called Google, followed by luck with Docker and Kubernetes adoption on the market, after they decided to rewrite from Python and Java respectively into Go, after Go heads joined their teams. Case in point, Limbo and Oberon-2, the languages that influenced its design, and authors were involved with.

I don't think that's the (only) reason Go became popular. The huge thing about Go is the runtime: it's the only language runtime available today, at least in any language with a large org behind it, that offers (a) GC, (b) fast start-up time, (c) static types, (d) fast execution, and (e) multi-threading.

This is a killer combination for any team looking to write code for auto-scalable microservices, to run for example on Kubernetes. Java is not great in this niche because of its slow startup time, relatively large memory overhead, and the need for warm-up before code actually starts executing fast (so scaling up and down has a very large cost for Java services). .NET has similar problems, and also a huge container size. Python is far too slow, and not typed. TypeScript is single threaded, and still has a pretty hefty runtime. OCaml doesn't have any large org behind it, is quite obscure syntax, and was still single-threaded at the time Kubernetes started. Haskell has similar issues, and is also large and slow starting. Rust, C++, C all require manual memory management.

So, it's no surprise that Go was used for Kubernetes services themselves, and it's no surprise that people designing for Kubernetes mostly chose to write their new stuff in Go. Go the language, with its antiquated design, is actually quite secondary to all of that. But Go's runtime is completely unmatched in this space.

Re: There is no memory safety without thread safety

#289

Post by astrange »

norir wrote: post #58
bombela wrote: post #23

Earlier quoted context omitted.

It took months to finally solve a data race in Go. No race detector would see anything. Nobody understood what was happening. It ultimately resulted in a loop counter overflowing, which recomputed the same thing a billion of time (but always the same!). So the visible effect was a request would randomly take 3 min instead of 100ms. I ended up using perf in production, which indirectly lead me to understand the data r…

It is very unfortunate that we use fixed width numbers by default in most programming languages and that common ops will silently overflow. Smarter compilers can work with richer numeric primitives and either automatically promote machine words to big numbers or throw an error on overflow. People talk a lot about the productivity gains of ai, but fixing problems like this at the language level could have an even bigg…

Swift traps on overflow, which I think is the correct solution. You shouldn't make all your numbers infinitely-ranged, that turns all O(1) operations into O(N) in time and memory, and introduces a lot of possibilities for remote DoS.

Re: There is no memory safety without thread safety

#290

Post by astrange »

singpolyma3 wrote: post #235

Honestly what I mostly want is to not have memory leaks. Which somehow stopped being a focus at some point

This is harder than it looks as soon as you start counting abandoned memory (stuff that's still referenced but not actually used.)
Post reply on HN