Bank of America's CEO says it's saved $2B per year by building its own cloud

Believe it or not most U.S. programmers don't live in the Bay Area.

For example if I can project steady growth or decline of my needs, it seems that I benefit less from the infrastructure-on-demand like AWS/Azure/GC

Also, depending on the size of the organization it seems that there at least several options on data centers:

  - have my own data center
  - collocate my hardware in other's data center
  - rent hardware
  - use infrastructure-on-demand + software-as-a-service

Many organizations deploy Pivotal's https://pivotal.io/industries/financial-services

or Redhat/OpenStack

So they are not actually 'building' the infrastructure management layer from scratch.

Also for investment banks (due to exchange connectivity) , telecoms (due to peering points, physical tower locations) always going to have their 'Front-office' applications in data centers that offer lowest latency for those apps.

Midoffice applications (or BOSS as they are labeled in telecom) can live in data centers reasonably distant (in terms of ping time) from the front office. So the infrastructure for those has a different cost profile.

Most of this applications touch trades or trade events that carry millions of dollars worth of contract value (not Facebook's likes or favorites... )

There is a lot of logging, reconciliation, post executing enrichment that's probably going on.

Finally, from my 2nd hand understanding,

Investment banks can have from 3K to 10K applications in production. With, perhaps 20% of them having more than million lines of code.

Some of those apps are built in their own proprietray languages developed specifically to manage contracts and lifecycle of contracts (Banks had 'smart contracts' DSL and interpreters since 90s).

https://news.efinancialcareers.com/uk-en/282097/slang-goldma...

They used off the shelf management software that they misconfigured. Getting VMs was so painful and they never worked correctly.

When I came back from paternity leave, they had gotten no response from their automated system and deleted all my VMs even though they could have seen heavy usage. If they had manually followed up they would have gotten an out of office email but they didn't.

Those are just two of the examples of how terrible it was to use their internal cloud.

You are always going to be five years behind if you do it internally. That might be a good cost calculation and I wasn't in a position to argue.

But, it meant innovation was terrible there.

In other examples, many airlines do lease airplanes, and even among the airlines that do own the planes, they still usually lease the engines.

As the GP said, there's a balance to be struck. Delta doesn't have the expertise or the economies of scale necessary to build their own engines from scratch, but they do have the ability to lease those engines and then have their own teams maintain them. Similarly, most companies don't have the know-how or the EoS to run their own data center, so instead they buy cloud services and then have a smaller team to maintain it. It's a win-win. Some companies like BofA might be big enough where they can run their own cloud and it be cheaper than using a cloud provider, but that's definitely not a common thing.

Anyone who has been in the industry since pre-cloud explosion knows that most people just rented bare metal and at the more complicated end they might purchase their own hardware and colocate.

And for those that own their servers, many lease them and HP or Dell or whoever maintains them on site if anything goes wrong.