Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is no memory safety without thread safety

ralfj.de

Post reply on HN
Print view View original

261–270 of 517 posts

Re: There is no memory safety without thread safety

#261

Post by 9rx »

stouset wrote: post #231
9rx wrote: post #190

Earlier quoted context omitted.

Already...? Said "explanation" was posted over an hour after the comment replied to here. If only Google put their weight into a watch, maybe you'd have one? Oh wait. They did! Google can't successfully turn their weight into much of anything. Go's success, if we can call it that, clearly happened in spite of Google.

Googlers aren’t expected to wear a Google-branded watch at work. They are expected to write go. Having an entire Google’s worth of programmers using your programming language isn’t exactly a minor influence.

> They are expected to write go.

Like who? Outside of Go itself, which is really more of a community project — albeit with the chief maintainers still on Google's payroll, almost nothing at Google is written in Go. In fact, Pike once gave a talk reflecting on why it didn't succeed in that space, and noted that it was the "Python and Ruby programmers" who actually ended up adopting it.

Google makes money selling services (i.e. Google Cloud) that run Kubernetes, Docker, etc. If it weren't for that, it is unlikely that Google would even be continuing to maintain it at this point. It was an interesting experiment, perhaps, but ultimately a failure within Google. As before, it was the Python and (probably most especially) Ruby communities that ended up leaning into it.

Which isn't surprising in hindsight. Go offered those who were using Python and Ruby a language that was in the same kind of vein, while solving many of the pain points they were experiencing with Python and Ruby (awful deployment strategies, terrible concurrency stories, trouble with performance, etc.) These developers were never going to use Haskell. They wanted Ruby with less problems.

And that's what Go gave them — at least to the extent of being better than any other attempt to do the same. Since it solved real problems people had, without forcing them into new programming paradigms, it was adopted. Choosing a technology based on starry-eyed fandom and arbitrary feelings might be how you go about navigating this world, but that doesn't extrapolate.

Re: There is no memory safety without thread safety

#262

Post by Waterluvian »

okanat wrote: post #243
Waterluvian wrote: post #238

Earlier quoted context omitted.

This is way outside my domain but isn’t the answer: yes, if the code is formally proven safe? Doesn’t NASA have an incredibly strict, specific set of standards for writing safety critical C that helps with writing programs that can be formalized?

There are safety recommendations / best practice standards like CERT. None of them will prevent you from making intentional looking but logically unsound memory unsafe operations with C and C++. The code can be very indistinguishable from safe code. The things that C and C++ allow you to do basically makes code written in those languages impossible to fully formally prove. Although there are subsets, the basic intege…

Thanks for explaining that. It really fills in a lot for me.

Re: There is no memory safety without thread safety

#263

Post by pizlonator »

dwattttt wrote: post #232
pizlonator wrote: post #206

Earlier quoted context omitted.

> If a language is "memory safe" but not "thread safe", is the result "the language is free from 'memory faults', unless threads are involved"? Yes. If a language is memory safe but not thread safe, then you can race, but the outcome of those races won't be memory corruption or the violation of the language's type system. It will lead to weird stuff, however - just a different kind of weirdness than breaking out of t…

> If a language is memory safe but not thread safe, then you can race, but the outcome of those races won't be memory corruption or the violation of the language's type system. By these definitions, doesn't that mean go is neither memory or thread safe? It looks like concurrent modification can result in memory corruption, e.g. the attempted access 0x42 example in the article

> By these definitions, doesn't that mean go is neither memory or thread safe?

Yes, with the caveat that you can't treat "memory safe" as a binary condition.

The strictest notion of memory safety is what I call GIMSO: "Garbage In, Memory Safety Out". I.e. there does not exist any sequence of bytes you could feed to the compiler that would result in a memory-unsafe outcome at runtime. Java aims for this. Fil-C does too. JavaScript also does.

But there are languages that I think it's fair to consider to be memory safe that offer escape hatches that violate GIMSO. Rust with `unsafe` is an example. C# with `unsafe` is another. Java if you include `sun.misc.Unsafe` (arguably it's not part of the language).

So I think if a language is memory safe, not thread safe, and the memory safety is gated on thread safety, then it's kinda fair to make statements like, "it's memory safe", if you have fine print somewhere that says "but the memory safety does not hold under the following kinds of races".

All of that said, I'd rather we just said that "memory safety" means what I call "GIMSO". But the ship has sailed. Lots of languages are called "memory safe" to mean something like, "you can get memory safety in this language if you obey certain idioms" - and in Rust that means "don't use unsafe" while in Go that means "don't race in certain ways".

Re: There is no memory safety without thread safety

#264

Post by tptacek »

socalgal2 wrote: post #257
tptacek wrote: post #168

Earlier quoted context omitted.

Right, the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term. People talk as if "memory safety" was a PLT axiom. It's not; it's a software security term of art. This is just two groups of people talking past each other. It's not as if Go programmers are unaware of the distinction you're talking about. It's literally the premise of the language; it's the basis for "sha…

> But: everybody understands that. Everybody does not understand that otherwise there would be zero of these issues in shipping code. This is the problem with the C++ crowd hoping to save their language. Maybe they'll finally figure out some --disallow-all-ub-and-be-memory-safe-and-thread-safe flag but at the moment it's still insanely trivial to make a mistake and return a reference to some value on the stack or any…

Again: if you want to make that claim about correctness bugs, that's fine, I get it. But if you're trying to claim that naive Go code has memory safety security bugs: no, that is simply not true.

Re: There is no memory safety without thread safety

#265

Post by tptacek »

codys wrote: post #251
tptacek wrote: post #162

Earlier quoted context omitted.

You need a non-contrived example of a memory-corrupting data race that gives attackers the ability to control memory, through type confusion or a memory lifecycle bug or something like it. You don't have to write the exploit but you have to be able to tell the story of how the exploit would actually work --- "I ran this code and it segfaulted" is not enough. It isn't even enough for C code!

The post is a demonstration that a class of problems: causing Go to treat a integer field as a pointer and access the memory behind that pointer without using any of Go's documented "unsafe.Pointer" (or other documented as unsafe operations). We're talking about programming languages being memory safe (like fly.io does on it's security page [1]), not about other specific applications. It may be helpful to think of th…

The thread you're commenting has already discussed everything this comment says.

If you've got concerns about our security page, I think you should first take them to the ISRG Prossimo project.

https://www.memorysafety.org/docs/memory-safety/

Re: There is no memory safety without thread safety

#266

Post by TheDong »

hu3 wrote: post #192
TheDong wrote: post #97

Earlier quoted context omitted.

Even if you use channels to send things between goroutines, go makes it very hard to do so safely because it doesn't have the idea of sendable types, ownership, read-only references, and so on. For example, is the following program safe, or does it race? func processData(lines The answer is of course that it's a data race. Why? Because `buf.Bytes()` returns the underlying memory, and then `Reset` lets you re-use the…

That code would never pass a human pull request review. It doesn't even pass AI code review with a simple "review this code" prompt: https://chatgpt.com/share/68829f14-c004-8001-ac20-4dc1796c76... "2. Shared buffer causes race/data reuse You're writing to buf, getting buf.Bytes(), and sending it to the channel. But buf.Bytes() returns a slice backed by the same memory, which you then Reset(). This causes line in proc…

> I mean, you're basically passing a pointer to another thread to processData()

And yet, "bytes.Buffer.ReadBytes(delim)" returns a copy of the underlying data which would be safe in this context.

The type system does not make it obvious when this is safe or not, and passing pointers you own across channels is fine and common.

> That code would never pass a human pull request review

Yes, that was a simplified example that a human or AI could spot.

When you actually see this in the wild, it's not a minimal example, it's a small bug in hundreds of lines of code.

I've seen this often enough that it obviously does actually happen, and does pass human code review.

Re: There is no memory safety without thread safety

#267

Post by crawshaw »

okanat wrote: post #246
crawshaw wrote: post #103

Memory safety is a big deal because many of the CVEs against C programs are memory safety bugs. Thread safety is not a major source of CVEs against Go programs. It’s a nice theoretical argument but doesn’t hold up in practice.

It depends on what threads can do. Threads share memory with other threads and you can corrupt the data structure to force the other thread to do an unsafe / invalid operation. It can be as simple as changing the size of a vector from one thread while the other one accesses it. When executed sequentiality, the operations are safe. With concurrency all bets are off. Even with Go. Hence the argument in TFA.

All bets aren’t off, we empirically measure the safety of software based on exploits. C memory handling is most of its exploits.

Show me the exploits based on Go parallelism. This issue has been discussed publicly for 10 years yet the exploits have not appeared. That’s why it's a nice theoretical argument but does not hold up in practice.

Re: There is no memory safety without thread safety

#268

Post by geodel »

stouset wrote: post #231
9rx wrote: post #190

Earlier quoted context omitted.

Already...? Said "explanation" was posted over an hour after the comment replied to here. If only Google put their weight into a watch, maybe you'd have one? Oh wait. They did! Google can't successfully turn their weight into much of anything. Go's success, if we can call it that, clearly happened in spite of Google.

Googlers aren’t expected to wear a Google-branded watch at work. They are expected to write go. Having an entire Google’s worth of programmers using your programming language isn’t exactly a minor influence.

> They are expected to write go.

This got to be a joke right. The only thing I hear is at Google no one likes Go. Most software is in C++, Rust, Java or Kotlin.

Re: There is no memory safety without thread safety

#269

Post by phire »

codys wrote: post #256
phire wrote: post #255

Earlier quoted context omitted.

The definition kind of changed. At the time Go was created, it met one common definition of "memory safety", which was essentially "have a garbage collector". And compared to c/c++, it is much safer.

That seems contrasted by Rob Pike's statement in 2012 in the linked presentation being one of the places where it's called "not purely memory safe". That would have been early, and Go is not called memory safe then. It seems like calling Go memory safe is a more recent thing rather than a historical thing.

Keep in mind that the 2012 presentations dates to 10 months after Rust's first release, and its version of "Memory Safety" was collecting quite a bit of attention. I'd argue the definition was already changing by this point. It's also possible that Go was already discovering their version of "Memory Safety" just wasn't safe enough.

If you go back to the original 2009 announcement talk, "Memory Safety" is listed as an explicit goal, with no carveouts:

"Safety is critical. It's critical that the language be type-safe and that it be memory-safe."

"It is important that a program not be able to derive a bad address and just use it; That a program that compiles is type-safe and memory-safe. That is a critical part of making robust software, and that's just fundamental."

https://youtu.be/rKnDgT73v8s?t=463

Re: There is no memory safety without thread safety

#270

Post by stickfigure »

tptacek wrote: post #245
moefh wrote: post #240

Earlier quoted context omitted.

I agree that there are two groups here talking past each other. I think it would help a lot to clarify this: > the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term So what is the actual meaning? Is it simply "there are no cases of actual exploited bugs in the wild"? Because in another comment you wrote: > a term of art was created to describe something complicated; i…

It's a contrived type confusion bug. It reads 42h because that address is hardcoded, and it does something that ordinary code doesn't do. If you were engaged to do a software security assessment for an established firm that used Go (or Python, or any of the other mainstream languages that do shared-memory concurrency and don't have Rust's type system), and you said "this code is memory-unsafe", showing them this exam…

I have recently come to the conclusion that everything I ever thought was "contrived" is currently standard practice in some large presently existing organization.
Post reply on HN