Earlier quoted context omitted.
> Spending serious engineering time to wrangle with the complexities of cloud orchestration is not something that should be taken lightly. Bare metal and datacenter orchestration is leaps and bounds more complex. You're paying for the abstraction.
If your scale is crazy, or your product doesn't allow you to use battle-tested pieces, then orchestration is complex in both cases. In most cases, managing software on bare metal is more complex in exactly one case: when engineers only know cloud abstractions.
1. managing compute clusters has gotten a lot easier but managing storage clusters and running good storage products for block & object storage on them is very very very far from a solved problem and quite frankly it sucks and is not fun.
2. planning out buying, installing, upgrading, patching, and retiring hardware, server/hypervisor OSs, takes waaaaaaaaay more engineering management skill and experience than 99.9% of companies have. Plus you probably have to fight for every dollar of investment against a board / investors. Even at the cloud provider we were constantly getting kneecapped by upper management not wanting to spend money on hardware.