61–70 of 71 posts
Earlier quoted context omitted.
Shouldn't Apple be the ones who really want to respond promptly? Why should we work around bugs in Apple's issue reporting system?
Usually big companies such as Discord give perks to the bug hunters who find bugs. Apparently Apple doesn't have that. There are probably people at Apple who won't admit that they have bugs, when every operating system has bugs, the code is too big to not create a single bug or exploit.
It's very easy to freeze a system as a non-root user; cause too many interrupts, consume too many resources, etc. Many kinds of infinite loop will lock a system hard. Hell, you can crash systems with too many packets . And it's very easy to cause ps to hang. Many different kernel syscalls hang / are blocking. Mostly you see this with kernel features dependent on a resource that doesn't resolve itself, like a stuck di…
The interesting thing here is that the described bug isn't just overloading the kernel with work or starving it of resources, it's something which seems completely innocent.
My naïve guess is that this is probably some sort of lock contention thing.
It's surprisingly easy to stumble into crash bugs when playing around with processes. I remember a decade or two ago I ran into a linux bug where the kernel would panic if a process was killed with an open descriptor on its /proc entries. That is: open /proc/$pid/something; kill -9 $pid #kernel crash We unfortunately discovered this when using fuser in a runscript to kill stale versions of a process, eg: sudo fuser -…
It was also unsurprisingly easy to crash a kernel a decade or two ago.
That was a wild time. Nobody talked about security back then. The idea that everything in our lives would eventually run over the internet just wasn't on people's minds.
Coincidentally, I also stumbled upon a way to make the kernel of Apple Silicon Macs panic and restart while developing the https://lowtechguys.com website. I distilled the problem in a repo so it can be reproduced with a single command: https://github.com/alin23/m1-panic I found it while on Monterey and reported it 2 times through Feedback Assistant, but it still happens on Ventura. NOTE: Don't try it without saving…
https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...
https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...
This uses the macro SLIST_REMOVE (https://github.com/apple-oss-distributions/xnu/blob/5c2921b0...) to remove an item from the linked list. If you look at the code to do this it's a pretty simple linked list traversal: https://github.com/apple-oss-distributions/xnu/blob/5c2921b0.... However, it doesn't have a check for the end of the list, so the item must be in the list, or it's going to walk right off the end and dereference a null pointer. In this case that's exactly what it ended up doing.
Coincidentally, I also stumbled upon a way to make the kernel of Apple Silicon Macs panic and restart while developing the https://lowtechguys.com website. I distilled the problem in a repo so it can be reproduced with a single command: https://github.com/alin23/m1-panic I found it while on Monterey and reported it 2 times through Feedback Assistant, but it still happens on Ventura. NOTE: Don't try it without saving…
The bug is in one of these two lines: https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... This uses the macro SLIST_REMOVE ( https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... ) to remove an item from the linked list. If you look at the code to do this it's a pretty simple linked list traversal: https://github.com/apple-oss-distr…
I know you're well versed in reverse engineering Darwin, and I'm reading your posts and trying to improve my skills in this daily, but this seems way over my skillset.
Did you debug this using KDK or m1n1? Do you have a setup always ready for debugging a Darwin kernel?
Earlier quoted context omitted.
The bug is in one of these two lines: https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... This uses the macro SLIST_REMOVE ( https://github.com/apple-oss-distributions/xnu/blob/5c2921b0... ) to remove an item from the linked list. If you look at the code to do this it's a pretty simple linked list traversal: https://github.com/apple-oss-distr…
That's.. wow, ok. How exactly did you end up in that source code at that specific line? I know you're well versed in reverse engineering Darwin, and I'm reading your posts and trying to improve my skills in this daily, but this seems way over my skillset. Did you debug this using KDK or m1n1? Do you have a setup always ready for debugging a Darwin kernel?
In this case I do not have a KDK for that build. In fact Apple has been unable to produce one for a couple of months, a inadequacy which I have repeatedly emphasized to them because of how critical they are for stuff like this. Supposedly they are working on it. Whatever; in lieu of that I got to figure out how good the tooling for analyzing kernels is these days, which was my real goal anyways.
For this crash log I downloaded the IPSW file for your build, 22A400. All of them get linked on The iPhone Wiki, e.g. https://www.theiphonewiki.com/wiki/Firmware/Mac/13.x. Once you unpack the IPSW (it's a zip file) there are compressed kernelcache files inside. Apple changed the format of these this year so most of the tooling breaks on it, but https://github.com/blacktop/ipsw was able to decompress them. Then I loaded it in to Binary Ninja, which apparently doesn't support them either but compiling this person's plugin (+166 submodules, and a LLVM & Boost build) gets it to work: https://github.com/skr0x1c0/binja_kc.
From there you can load up the faulting address from the crash log and see what the function looks like. In this case, a bunch of junk has been inlined into it but there's a really obvious and fairly unique string reference for "invalid knote %p detach, filter: %d". From there, you can compare it against the actual source code to see which one matches the "shape" of the function you're looking at. I happened to also pull up an older kernel which did have a KDK available and then compared its assembly to the new one to match it up to ptsd_kqops_detach. The disassembly of the crashing code is obviously doing a linked list walk so you can figure out exactly which line it is from that.
If I wasn't lazy I might also fire up a debugger to see why the function had walked off the end of the list but without KDKs things get pretty bad, not that they're very good to begin with. I don't have a m1n1 setup (I should probably do this at some point) and the things I do have, like remote debugging or the VM GDB stub, are not really worth suffering through for a Hacker News comment.
Earlier quoted context omitted.
That's.. wow, ok. How exactly did you end up in that source code at that specific line? I know you're well versed in reverse engineering Darwin, and I'm reading your posts and trying to improve my skills in this daily, but this seems way over my skillset. Did you debug this using KDK or m1n1? Do you have a setup always ready for debugging a Darwin kernel?
In theory I have all of those, but currently I have none, so it's manual work. Your best friend in diagnosing a kernel crash is a KDK. If you have one that matches your build, it will have symbols in it. With a little bit of math you can take the backtrace in the crash log and slide it appropriately to match the binary. Personally I use LLDB for this. Here's an example of what this looks like on an x86-64 kernel (App…
I was in the process of trying to get I²C working through the built-in HDMI port of the Apple Silicon Macs (the one containing the MCDP29xx HDMI-to-DP converter chip) and been hitting a lot of dead ends while looking at kexts and opaque firmware blobs. This is going to help a lot as the KDK seems to contain logging messages related to DDC that I've never seen before.
I also found SIP disabled + Frida very useful for debugging without going through the KDK/m1n1 route. Not sure if it also helps with kext code though, I mostly used it for SkyLight and other private frameworks, but it's very nice to be able to also alter the code while it is running in realtime, or sometimes simply log specific function calls with argument value to get an idea what action causes which code to run.
Earlier quoted context omitted.
In theory I have all of those, but currently I have none, so it's manual work. Your best friend in diagnosing a kernel crash is a KDK. If you have one that matches your build, it will have symbols in it. With a little bit of math you can take the backtrace in the crash log and slide it appropriately to match the binary. Personally I use LLDB for this. Here's an example of what this looks like on an x86-64 kernel (App…
Saagar, thank you so much! This is priceless! I was in the process of trying to get I²C working through the built-in HDMI port of the Apple Silicon Macs (the one containing the MCDP29xx HDMI-to-DP converter chip) and been hitting a lot of dead ends while looking at kexts and opaque firmware blobs. This is going to help a lot as the KDK seems to contain logging messages related to DDC that I've never seen before. I al…
Earlier quoted context omitted.
I just tested right now, ⌘C copies the string "1e20" and ⌘V pastes that same string. So yeah it looks like it was fixed.
I also just tested now and reproduced the bug. The key thing is that you are pasting back into the calculator – which presumably is just stripping letters. The behavior is maybe a bit surprising, but I could also see it being defensible. You can't type "1e20" into the calculator, so why would you be able to paste it in? Outside of plain text, I don't think clipboard operations are necessarily expected to be reversibl…