Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is no memory safety without thread safety

ralfj.de

Post reply on HN
Print view View original

451–460 of 517 posts

Re: There is no memory safety without thread safety

#451

Post by zozbot234 »

tptacek wrote: post #443
zozbot234 wrote: post #437

Earlier quoted context omitted.

There's no "revelation" here, it's always been well known among experts that Go is not fully memory safe for concurrent code, same for previous versions of Swift. OP has simply spelled out the argument clearly and made it easier to understand for average developers.

It's made what would be a valid point using misleading terminology and framing that suggests these are security issues, which they simply are not. "One could easily turn this example into a function that casts an integer to a pointer, and then cause arbitrary memory corruption." No, one couldn't! One has contrived a program that hardcodes precisely the condition one wants to achieve. In doing so, one hasn't even demo…

> misleading terminology and framing that suggests these are security issues

Could you quote where exactly OP has misleadingly "suggested" that these concerns lead to security issues in the typical case?

> attacker control of the data, and attacker ability to place controlled data somewhere advantageous to the attacker

Under this definition the Rowhammer problem with hardware DRAM does not qualify as a genuine security concern since it inherently relies on fiddly non-determinism that cannot possibly be "controlled" by any attacker. (The problem with possible torn writes in concurrent Go code is quite similar in spirit; it's understood that an actually observed torn write might only occur rarely.) Needless to say there is a fairly strong case for addressing these problems anyway, as a matter of defence in depth.

> correctness advantages of Rust

Memory safety in OP's sense is not exclusive to Rust. Swift has it. Even Java/C# cannot access arbitrary memory as a result of torn writes. It would be more accurate to say that OP has identified a correctness issue that's apparently exclusive to Go.

Re: There is no memory safety without thread safety

#452

Post by ralfj »

kazinator wrote: post #10

This is false as a generality. A memory safe, managed language doesn't become unsafe just because you have a race condition in a program. Like, say, reading and writing several related shared variables without a mutex. Say that the language ensures that the reads and writes themselves of these word-sized variables are safe without any lock, and that memory operations and reclamation of memory are thread safe: there a…

> The rest is your bug; the variable values coming out of sync with each other, not maintaining the invariant among their values.

If the language and its runtime let me break their invariant, then that's their bug, not mine. This is the fundamental promise of type-safe languages: you can't accidentally break the language abstraction.

> It could be the case that a thread-unsafe program breaks a managed run-time, but not an unvarnished truth.

I demonstrated that the Go runtime is such a case, and I think that should be considered a memory safety violation. Not sure which part of that you disagree with...

Re: There is no memory safety without thread safety

#453

Post by bombela »

Yoric wrote: post #132
9rx wrote: post #124

Earlier quoted context omitted.

> Plenty of very interesting ideas Is there? When you get down to it, it is really just a faster Python. Which is exactly what it was said to be when it was released. Their goal was to create a "dynamically-typed" language that was more performant. It is likely that it wouldn't have had a static type system at all if they figured out how to achieve on the performance end without needing types. You can tell who is clu…

Two examples of interesting ideas: - using zero values as an optimization mechanism; - (non-)pointers and passing self by copy. I mean, I hate both mechanisms, but intellectually, I find them quite interesting. Also, I'd not classify it as a faster Python. It's more of a cousin of Obj-C if the authors of Obj-C had fallen in love of Erlang instead of Smalltalk.

Those ideas where already present in C/C++ decades prior. BSS program area. And passing struct by value vs pointer.

Re: There is no memory safety without thread safety

#454

Post by jchw »

bombela wrote: post #441
jchw wrote: post #234

Earlier quoted context omitted.

Aside from the type system bypass described in the article though, this is basically no different from the status quo for virtually all languages with free threading that aren't Rust. I argue that while everyone is fallible, experienced programmers usually don't make this sort of mistake directly, because they are usually well aware that sharing pointers over a closure on another thread is a recipe for disaster. Inst…

Yes experienced programmers won't make the obvious mistake... until they do because the distance from the source of the bug to it's manifestation is too great to notice until it fails in production.

Look, this is pointless. I'm not learning anything new when you tell me that it can and will happen. How will it happen and how much will it happen?

Hence linking to Uber's case study on the issue. The answer? Not that much.

Uber started performing race detection in production over a 6 month period and found 2,000 different race conditions. Ouch, that sounds horrible!

But wait, we're talking about 50 million lines of Go code and 2,100 services at the time of that writing. That means they were seeing approximately 1 race condition per 25,000 lines of code and about 1 race condition per service. That actually lines up pretty well with my experiences. Although I haven't had a production outage or serious correctness issue caused by a race condition in Go, I have seen probably about one or two race conditions that made it to production per service. I reckon those codebases were likely somewhere between 10,000 and 25,000 lines of code most likely, so not so far off of the scale.

But again it doesn't always lead to a serious production outage, it's just that simple. It could be worse too (could corrupt some data and pollute your production database or something, in the worst case) but usually it's better (wonky behavior but no long-term effects, maybe the service periodically crashes but restarts, leading to some dropped requests but no long term downtime.) Uber has no doubt seen at least some Go data races that have caused actual production outages, but they've seen at least 2,000 Go data races that haven't, otherwise they would've probably been caught before the race detector caught them, Go dumps stacktraces on crash. That has to tell you something about the actual probability of causing a production outage due to a data race.

Again, you do you, but I will not be losing sleep over this. It is something to be weary of when working on Go services, but it is manageable.

Re: There is no memory safety without thread safety

#455

Post by esoterae »

HAL3000 wrote: post #423
ralfj wrote: post #368

Earlier quoted context omitted.

> While you're wondering why I keep claiming Go is a memory-safe language, you can also go ask the ISRG, which says the same thing I am at And yet Go violates the definition they give -- it doesn't prevent out-of-bounds accesses. (And just to be sure we're talking about the same thing, I'm specifically talking about Go here. All the other languages on their list are actually memory safe, as far as I know.) > you have…

> So your definition of memory safety includes some notion of "plausible" and "realistic"? Neither https://www.memorysafety.org/docs/memory-safety/ nor Wikipedia have such a qualification in their definition. It would help if you could just spell out your definition in full, rather than having us guess. This is a strawman argument, you're arguing semantics here. You're a smart person, so you know exactly what he mean…

Did you not notice how this started over someone saying "That's not the definition of memory safety" and then prevaricating about the bush when asked to provide their definition? Your theory that this is an argument over semantics is correct, but not fully understood.

Re: There is no memory safety without thread safety

#456

Post by zozbot234 »

jchw wrote: post #454
bombela wrote: post #441

Earlier quoted context omitted.

Yes experienced programmers won't make the obvious mistake... until they do because the distance from the source of the bug to it's manifestation is too great to notice until it fails in production.

Look, this is pointless. I'm not learning anything new when you tell me that it can and will happen. How will it happen and how much will it happen? Hence linking to Uber's case study on the issue. The answer? Not that much. Uber started performing race detection in production over a 6 month period and found 2,000 different race conditions. Ouch, that sounds horrible! But wait, we're talking about 50 million lines of…

Identifiable "wonky" behavior and periodic crashes seem like a very real issue to me. This wouldn't fly for any mission-critical service, it's something that demands a root cause analysis. Especially since it's hard to be sure after the fact that no data has been corrupted somehow or that security invariants have not been violated due to the "wonky" behavior.

Re: There is no memory safety without thread safety

#457

Post by tptacek »

zozbot234 wrote: post #451
tptacek wrote: post #443

Earlier quoted context omitted.

It's made what would be a valid point using misleading terminology and framing that suggests these are security issues, which they simply are not. "One could easily turn this example into a function that casts an integer to a pointer, and then cause arbitrary memory corruption." No, one couldn't! One has contrived a program that hardcodes precisely the condition one wants to achieve. In doing so, one hasn't even demo…

> misleading terminology and framing that suggests these are security issues Could you quote where exactly OP has misleadingly "suggested" that these concerns lead to security issues in the typical case? > attacker control of the data, and attacker ability to place controlled data somewhere advantageous to the attacker Under this definition the Rowhammer problem with hardware DRAM does not qualify as a genuine securi…

I quoted directly from the article.

Re: There is no memory safety without thread safety

#459

Post by zozbot234 »

tptacek wrote: post #457
zozbot234 wrote: post #451

Earlier quoted context omitted.

> misleading terminology and framing that suggests these are security issues Could you quote where exactly OP has misleadingly "suggested" that these concerns lead to security issues in the typical case? > attacker control of the data, and attacker ability to place controlled data somewhere advantageous to the attacker Under this definition the Rowhammer problem with hardware DRAM does not qualify as a genuine securi…

I quoted directly from the article.

To use your definition, that quote is clearly making a point about correctness, not necessarily about real-world security.

Re: There is no memory safety without thread safety

#460

Post by sophacles »

brabel wrote: post #450
sophacles wrote: post #422

Earlier quoted context omitted.

>Rust is objectively harder to write than Go (or any other GC-language) because it exposes more concerns for the programmer to take care of. comparing apples to apples: Once you get a tiny bit of experience, almost all of that goes away. The common patterns and idioms in the language allow you to write whole programs without ever thinking about lifetimes or memory allocation or anything else different from the gc lan…

> I think people who say this haven't gotten much experience in rust. In my experience they spent a week trying to learn rust and decided to stop and compare it to their years of other languages and paradigms. I have written Rust for around 6 years now.

That doesn't say much about experience. Ive written ruby at a rate of a few dozen lines/year, for the 20 years.

I guess I could say I've written ruby for 20 years... But someone full-time in ruby for only a year would likely be significantly better at the language than I am (i am bad at it).

Post reply on HN