Live data from Hacker News

Bank of America's CEO says it's saved $2B per year by building its own cloud

businessinsider.com

111–120 of 321 posts

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#111

For what it's worth, I really don't like AWS - aside from their S3/Route53/CloudFront products. There is a layer of abstraction that is so minimal that you may as well roll out the services yourself & save a ton of money. Of course, if you like that layer, then that's fine too.

If you look beyond that, AWS offer layers of services that you might not realize existed.

Cloud DB with no worry about physical hardware (in addition to S3 buckets which is mostly for files). lambdas when you don't want to manage a server, cloudformation when you don't want to manually start your stack by clicking, API gateway for quick ways to create APIs without worrying about stuff. And powerful orchestration layer that connects all of these together.

In fact, a lot of these services are bootstrapped on AWS itself. If you know your way around AWS, you can really find a sweet spot where it's cheap scalable and mostly worry free. The problem is that AWS domain knowledge in itself is _deep_ and not many people know enough about AWS to avoid large amount of pitfall and inefficiencies that a knowledgeable people can quickly spot and fix/alternate.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#112
post #34

I think a strong argument can be made either way. There's definitely financial savings to be had by doing it in-house (either literally or utilizing existing data centers as here). But what I've experienced is that companies are often attracted to Big Cloud™ not just for financial reasons but to flatten/simplify their corporate reporting structure. It is the ultimate delegation. This isn't simply about having fewer s…

> In this case BoA are saving money but in exchange retain additional layers of staffing, that require oversight, and ultimately decision makers willing to take risks. I think some companies, like BOA, look at this as the safer alternative. We all assume that large Cloud providers are bulletproof. That's just because we haven't seen them taken down yet. Someday we will. Just wait until there's another major world con…

> In this case BoA are saving money but in exchange retain additional layers of staffing, that require oversight, and ultimately decision makers willing to take risks.

I've been at multiple Fortune 50 companies at multiple levels. Many times these types of infrastructure building are just forms of empire building and an attempt to embed ones' services in the organization at a very deep level. You convert one set of costs (AWS/GCP/Azure bills) for another set of costs (Infrastructure + Payroll). You also convert one set of headaches (CSP overcharging/auditing) for another (prima donna internal infra engineers.)

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#116
post #66

Earlier quoted context omitted.

> This isn't simply about having fewer staff (although that can reduce internal politics and inefficiencies), but making fewer decisions and more importantly reducing the potential for making the wrong decisions This is a myth directly from cloud providers marketing campaigns: that using the cloud simplifies infrastructure to the point that you don't need as much and/or as qualified staff. In reality it is quite the…

Yeah but you don’t need data center engineers, real estate specialists. Power specialists. Etc.

If you build your own DC you would outsource that to a consultancy or contractor.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#118
I have a bullet proof plan to earn a $10m bonus in my two years as CTO at :

1. Identify retarded firm that uses AWS or Azure like money grows on trees AND without too much AWS lock-in

2. Join as CTO and move infra to OVH

(2.1 Convince OVH to not irritate international customers by making my browser show "Translate this page?" for EVERY SINGLE page, even on us.ovhcloud.com, because they HAVE to show their French lineage at all times instead of just having English pages in English! They've already improved their Customer Service a lot, so I have great hopes)

3. Reduce server costs by 4X

4. Profit!

Seriously. Unless you really need auto-scaling or one of the other unique features of AWS / Azure products, you don't need to pretend you're the next Facebook or Netflix.

Also seriously: de-cloudification is a great business idea.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#119

> The bank, which has a $10 billion annual tech budget Am I the only one completely befuddled by this number? What the fuck are they doing with these money and 200k servers? These are facebook numbers. For a bank. What?

Might well have small systems at branch level that they are counting as servers.

Apparently BOA has 4344 branches so the local infrastructure costs will add up both for branch kit and the networking costs.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#120
I used to work for BofA as a quant in their Charlotte HQ. Their cloud decision is the least surprising. You really have to see it from their pov. My boss used to say - "Charlotte is a 2-horse town. You either work for Bank of America, or you walk across the street to Wells Fargo"! Compared to these 2, the other companies are much smaller along most axes (market cap,employee count, or just sheer heft). There's an uptown walkway( like a private overhead glass tunnel) that safely escort bank workers from downtown to bank without coming into contact with riffraff :) Its just a whole new level of planned design. Imagine if all the FAANGs had an interconnected private glass tunnel walkway that looked down upon the unfortunate denizens of MV/PA/SF,while the chosen ones swiftly segwayed from FAANG to FAANG whilst checking in their latest git commit into kubernetes or whatever it is you guys do :) That's what it was like. Quite unreal.

But all this clout leads to a rather head in sand mindset on most strategic items, like technology choice, programming language choice, cloud choice (or non-choice in this case), version control choice etc. Everything was done in-house, in the most boring safest way possible tech ( mostly Java about 3 versions behind, some strange python lib where any function call was automatically logged!, and Excel & Matlab all over the quant land. I mostly sftp-ed financial data from some Quartz cloud...felt very quaint to do these sort of things in 201X. All laptops were locked down windows dell boxes on which you couldn't install anything, & ran some strange norton antivirus which hogged all the memory. My interview itself was so old fashioned. I thought since it was a Quant job, I'd get questions on math & finance. They trotted out their "chief developer" who wanted to know how to model a chair with 4 legs using Java OO. You know the 1990s Grady Booch garbage full of UML, with parent Table class & child Table & Leg Class & friend function & all that jazz. I was like Jesus this regressive inheritance based OO shit is still alive! Its a deeply old fashioned slow moving place. Very large IT budget with pretty much half of Charlotte working in some capacity for the bank. So yeah, if you had all the personnel & all the money, why wouldn't you build your own cloud. You are paying for all these people anyway, might as well give them something to do. Of all the employers I've worked for in my lifetime, this was the one place where I was personally asked NOT to work so hard, because I stayed at my desk after 5:30 pm.

(Sorry I have a regular 4 digit HN account but the bank doesn't like it if you talk about them. One of their lawyers once tracked me down because I mentioned some harmless datapoint about a technical problem I had worked on.)

Post reply on HN