I think the mistake is often made by comparing primitives. E.g. running my own RAID vs S3. Colo traffic vs AWS traffic. But what about comparing the whole ecosystem? Can you provide a self hosted granular access permission to your RAID? How hard is it to configure and maintain? Will your colo deflect a DDOS attack? When you run your own services, you have to reinvent so much it doesn’t seem to be worth it.
This is the second-level mistake engineers commonly make:
The right questions isn't "Can you do X". Give engineers enough time and resources and they can usually come up with a solution to do X.
The real question is "How much time and resources need to be invested to accomplish X at a satisfactory level?"
And the third-level mistake is to assume that getting something to work once is the finish line. In practice, getting something to work once is just the beginning. Getting it to a maintainable, well-documented, repeatable state is a lot more work.
Cloud services make all of this effort disappear. Type a few commands and it's good to go. Now you can take all of the engineering hours that would have gone into the DIY version and allocate them to working on the company's product instead of reinventing architecture that you could have simply paid for.
Good engineers are scarce and expensive. Using them to reinvent infrastructure that can be trivially purchased for a nominal amount is a terrible move most of the time. Even when it does make sense, the right move is to build the prototypes on AWS and then consider transitioning to self-hosted later if the numbers work out.