121–130 of 146 posts
Earlier quoted context omitted.
> So another win for being able to read arm assembly. Yes, though that weird stuff with dollars in it is not normal AArch64 assembly! The article could have mentioned the "stack moves once" rule.
> Yes, though that weird stuff with dollars in it is not normal AArch64 assembly! See the AT&T vs Intel syntax since you aren't familiar with assembly: https://en.wikipedia.org/wiki/X86_assembly_language#Syntax
Earlier quoted context omitted.
Yeah but we have codegen bugs in .NET as well. The biggest difference that stood out to me in this write up, is we would have gone straight for “coredump” instead of other investigation tools. Our default mode of investigating memory corruption issues is dumps.
Sure, I have experienced them, e.g. once in 2006 using IBM's JVM implementation with Websphere. However it is probably not as problematic due to the way Go allows for Assembly being used directly. While the JVM and CLR don't allow for direct access to Assembly code, Go does, thus I assume expecting safepoints everywhere is not an option, as any subroutine call can land on code that was manually written.
(Well technically there is a way to inject assembly without the function call overhead. That's what https://pkg.go.dev/runtime/internal/atomic is doing. But you will need to modify the runtime and compiler toolchain for it.)
Even Java, as widespread as it is, I have made half-a-dozen reports. None in the last several years, though.
Better testing? The sheer scale of software being produced?
I find it interesting, how rare it has become to find s compiler bug. For me, at least, it used to be a regular event. Even Java, as widespread as it is, I have made half-a-dozen reports. None in the last several years, though. Better testing? The sheer scale of software being produced?
But definitely, better engineering and QA practices must also help here.
Always adjust your stack pointer atomically, kids.
I guess those that wrote the preemption were on X86 where this doesn't happen thanks to variable length instructions being able to hold the constant and thus relied on the code-gen to do it atomically, then the ARM port had an automatic "split" from a higher level to make things "easy" thus giving us this bug. Nobodys fault really, but bad results ensued.
(You cannot express relaxed atomics in golang, but you could technically add support in the compiler for use in the runtime code)
That's an incredible find and once I saw the assembly I was right along with them on the debug path. Interestingly it doesn't need to be assembly for this to work, it's just that that's where the split was. The IR could've done it, it just doesn't for very good reasons. So another win for being able to read arm assembly. Unsure if this would be another way to do it but to save an instruction at the cost of a memory a…
Earlier quoted context omitted.
It becomes fun when you narrow down to the solution. Before that it's hell. I don't think I'd be allowed spend weeks to debug something like this. Credit to Cloudflare's PMs.
Apparently they have a "unexplained crashes must have an explanation determined" policy ever since there was a trend of uninvestigated unexplained crashes that were canaries in the mine for a security issue. https://blog.cloudflare.com/however-improbable-the-story-of-... > But [the Cloudbleed sensitive information disclosure security incident] wasn’t the only consequence of the bug. Sometimes it could lead to an inva…
As an aside, this is the type of a problem that I think model checkers can't help with. You can write perfect and complicated TLA+/Lean/FizzBee models and even if somehow these models can generate code for you from your correct models you can still run into bugs like these due to platform/compiler/language issues. But, thankfully, such bugs are rare.
Great find and writeup. As an aside, this is the type of a problem that I think model checkers can't help with. You can write perfect and complicated TLA+/Lean/FizzBee models and even if somehow these models can generate code for you from your correct models you can still run into bugs like these due to platform/compiler/language issues. But, thankfully, such bugs are rare.
For the implementation, you can use certified compilers like CompCert [1], but:
- you still have to show your code is correct
- there are still parts of CompCert that are not certified