An AI won't replace the CEO, because the AI can't take responsibility and step down for decisions the board of directors made. Now having the AI replace the board of directors is a more interesting proposition...
We can definitely destroy the version of AI CEO that made the bad decisions and replace it with a „better“ version.
A CEO has a fiduciary duty that an AI cannot fulfill. Although we variously like to beat up on shitty CEOs around here (rightfully so), a corporate officer of any title requires an ability for cognition, reflection, and abstraction that an AI does not possess.
That might be a bit of a technical argument in a way, but at the end of the day AI doesn’t change the fact that you can’t hold a computer responsible. It’s a machine.