Earlier quoted context omitted.
It's just very hard to write such systems in C/C++. Even all these Rust versions are segfaulting and panicking quite a lot. So many corner and edge cases that can be found in Python code, and the memory handling is also hard. The author of Zuban started writing it back in 2020 or 2021, so it took him more than 4 years to complete it. And he is the author of Jedi, so he had prior experience already.
I get why they'd panic, but why have enough segfaults that you noticed? So, I went and read the code. Zuban seems to have a bunch of scary "I'm not sure if this is correct" unsafe blocks, which to me would be a red flag. I mean, it's better that there's a comment expressing the doubt, but my experience is that if you're not sure whether it's correct, it's probably not correct.
For anyone reaching for unsafe, there are in many cases either an existing API (split_at_mut comes to mind). For others, using zero-copy or bytemuck instead of unsafe is a good idea too.
None of that is to say "never write unsafe", unsafe existing is pretty much one of the reasons for Rust to be :)