(This is neither direct disagreement nor agreement, but a sort of elaboration of my own.)
The practical semantics of a panic become increasingly ill-defined as the number of threads in the program and the depth of their stacks increase. I emphasize "practical" because mathematically, no matter how many threads you have or how deep you get, the semantics are perfectly well defined; it's going to do whatever it's going to do. But those mathematical semantics get more and more complicated as the program grows. If you call X() and it panics, its behavior depends on its stack above. If that X() is called in a "server" thread and it ends up terminating it (or restarting it, or anything else), it may impact any number of other threads in weird ways, even if you "handle" the panic in some manner, e.g., even if you "clean up the locks" on the way up the panic handler that still doesn't mean the program's in a "clean state".
When a programmer goes to write the aforementioned X(), it's impossible in general for that programmer to know what their "panic environment" is going to be. (In specific, it may be an unexposed/private method that does have a good idea what environment it will be called in, but in general a function does not know.) Working via panic handlers shares the responsibility between the caller and the callee in a difficult-to-describe manner. At small scales this is completely neglectable, but as you scale up, mismatches between the programmer of X() and the user of X() very gradually start accumulating, and interacting with the other mismatches, and at some point you get to the point where some junior programmer doesn't fully understand the complicated maze and just starts bashing on code until it superficially seems to work, and then you're in real trouble.
By contrast, the contract with error-type returns like Result or Either or whatever has a clean contract between the caller and the callee; the callee is responsible for reporting failure, and the caller is responsible for dealing with it. This kind of contract does not compose up into a complicated "panic environment", because it does not cross the stack frames like a panic/exception can. The frames don't mix with each other.
In theory it may be possible to completely exclude exceptions; in practice right now we don't really know how to do it 100% in practical languages, so regardless of what I wrote and regardless of what you may like, you will have some sort of "panic environment". However, I think you are in general better off to try to avoid complicating it as much as possible, because it gets exponentially complicated. I do mean exponentially as in 2^x and not x^2 as it is commonly abused. But the x is fairly small, which makes it easy to fail to notice. Also if you know your program is going to stay reasonably small it means it's a valid choice to just not care and use it anyhow. But if you have any reason to think your program may scale, I'd suggest trying very hard to stick to error-type handling as much as possible, as you only pay that exponential term as you add things into your panic environment.
(Also, I know that "using errors" doesn't fix the problems I mentioned, as I myself have had plenty of days where my "error based" program terminated a critical internal server and made the program "go all weird". I'm just saying it makes it better to not add the complications of a complicated panic-handling environment in addition to the already-unavoidable other tasks you have to do.)