Live data from Hacker News

Bank of America's CEO says it's saved $2B per year by building its own cloud

businessinsider.com

311–320 of 321 posts

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#311

I think a strong argument can be made either way. There's definitely financial savings to be had by doing it in-house (either literally or utilizing existing data centers as here). But what I've experienced is that companies are often attracted to Big Cloud™ not just for financial reasons but to flatten/simplify their corporate reporting structure. It is the ultimate delegation. This isn't simply about having fewer s…

> This isn't simply about having fewer staff (although that can reduce internal politics and inefficiencies), but making fewer decisions and more importantly reducing the potential for making the wrong decisions This is a myth directly from cloud providers marketing campaigns: that using the cloud simplifies infrastructure to the point that you don't need as much and/or as qualified staff. In reality it is quite the…

If you already have IT staff in-house it ends up being a duplication of roles and an added layer of complexity. I get some servers and/or resources can benefit from autoscale. USGS gets slammed after a major earthquake so having autoscale capability is important. Or you want to co-locate some servers (payroll) for disaster recovery purposes. There are some legitimate uses cases, but often cloud just feels like renting servers that you could own and manage in-house.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#312

Earlier quoted context omitted.

Bay area or NYC sure, that's like being in London, but what about the Midwest, or any of the places where you can easily set up an IT shop without paying a premium for space?

how will you attract the talent?

- Get your own bedroom. - 30 minute commute. - Not everyone you meet will be in tech. - You'll be rich compared to everyone else in town. - Partner with local academic institutions to offer recognized research projects and training. 20% time or 3 month project stints etc. Adjunct professor ships for people with an existing research record. - Throw in X free flights to east/west coast, home town etc.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#314

Earlier quoted context omitted.

I guess I have a naive view. I think of "buy" as something which is a capital expenditure, can be depreciated, etc. while "rent" are things which fall under operational expenditure. With that in mind, I can see support contracts as an operational expenditure. But if "over half of industrial robot purchases in North America have been made by automakers" then that's a lot of capital expenditure. I never got the feel th…

Many of these industrial systems are built from reusable components. In my experience at a Tier 1 OEM for the US domestic auto market, a line could be torn down and re-purposed to another line by adding or removing physical components, modifying the physical layout where appropriate, and then adjusting the PLC programming. An assembly line isn't really a "product," per se. It's more of amalgamation of several product…

[deleted]

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#315

Earlier quoted context omitted.

This must be a jab, but obviously yes. Last year they had $28 billion in net income. With that kind of money you can hire the best in the world to build literally whatever you want. They could go build a space shuttle if they felt like it - probably without much trouble. In the unlikely situation that it didn't work, they'd pay someone else to fix it or re-do it. The least likely (albeit not unprecedented) end situat…

Boeing can't even make an airplane without trouble and you think BoA could churn out a space shuttle? The space shuttle program was estimated to have cost $200B.

Boeing has made literally tens of thousands of airplanes "without trouble". One debacle doesn't mean they "can't even make an airplane".

"Space Shuttle" was rhetorical, but yeah, they could churn out a space shuttle. They could buy SpaceX and use their engineers to do it. BofA has $2.3 TRILLION in assets.

The point is you can buy expertise. It's not like a bunch of bank tellers coded up a cloud platform. Nor would you have a regional branch manager running an aerospace program. You'd hire people that know how to do those things the same as any other expert at any other company.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#316
post #156

Earlier quoted context omitted.

That's not a comparison of external cloud vs. self hosted though. It is a comparison of different operating models with respect to change control. Either one can exist in both external vs. self-hosted infrastructure. I've been in places where the engineering team plugs in an old PC under a desk somewhere, gives it a public IP address of there's the production server. That's the self-hosted equivalent of the engineeri…

Change control and service delivery time-frames are two different things. When I left BOA in 2017, you could get most production changes implemented with only two days of lead time from a formal change control perspective. But, requesting a new Virtual IP on a load-balancer could easily take two weeks, just for every layer of bureaucracy to wet its beak. And it was impossible to request something so basic without an…

Most production changes require a second set of eyes, sometimes from a particular team, but it's all "just" code review. You put your change in the team's queue, their oncall engineer reviews it the same day, you land the change and it gets executed automatically. Most have implemented namespacing so that changes that only affect your own team's stuff can be approved within your team.

This is all on owned hardware. The difference is that we're a SWE driven company (corporate IT is off in its own world, run in the more traditional way, but they don't touch engineering's production datacenters). Infrastructure teams provide APIs, not JIRA forms.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#317

Earlier quoted context omitted.

Because of the capital and talent required to build a competitive offering. Even large companies like IBM and Oracle have failed to keep up. The big 3 clouds have built a nice moat and can charge whatever they want. They still compete on prices but it's clear they're colluding on network fees at least.

IBM and Oracle are really not in the same class as AWS, Azure and GCP, when it comes to investing in cloud infrastructure. They’re bit players at best: http://www.platformonomics.com/wp-content/uploads/2018/05/cl... From this article: https://www.platformonomics.com/2018/05/follow-the-capex-sep...

Yea, that's the point. If they can't keep up then nobody else can.

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#318

Earlier quoted context omitted.

how will you attract the talent?

- Get your own bedroom. - 30 minute commute. - Not everyone you meet will be in tech. - You'll be rich compared to everyone else in town. - Partner with local academic institutions to offer recognized research projects and training. 20% time or 3 month project stints etc. Adjunct professor ships for people with an existing research record. - Throw in X free flights to east/west coast, home town etc.

there’s 30 minute commutes in san diego, la and sf if you live in the right areas. what about weather? cali has really nice weather. schools / doctoral candidates / etc are basically useless in the real world. if i’m making enough money the free flights are useless. nothing you said here makes me want to move to the middle of the US. exactly what i’m looking at is the lack of opportunity. say i decide to move for some job and hate it. now what? pick up the family and move again?

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#319

Earlier quoted context omitted.

the difference is CapEx is dropped all at once and OpEx is over time. so the accounting is not the same

You can use financing arrangements to avoid dropping the money all at once -- for example: A service provider offers you a monthly subscription or "lease-to-own deal" where you sign a contract to buy the software or hardware in exchange for monthly payments to be made over a period of time; that could be indefinite, or you become the owner of the hardware after the end of its useful life. Or just go to a bank, and th…

in the case of computer equipment that needs replacement very 4-5 yrs (or less depending on compute and other reqs) you’re perpetually in a lease and this starts to become opex

Re: Bank of America's CEO says it's saved $2B per year by building its own cloud

#320

Obviously small businesses don't have the option of building their own cloud, but there is a middle ground: use public clouds, but avoid coupling yourself to them. Most cloud providers provide servers that can be treated like a barebones Linux server, and that's all you really need to be productive with most server applications.

Then just use a colo center. It's worked for us for close to 20 years. Last time someone had to perform hardware maintenance was at least 3 months ago. If you're small enough (20-30 servers for us) the things mostly run themselves. Any other overhead we have would be present if we were just using something like EC2. The bulk of our hardware is 10 years old. We have some servers that are 17 years old and still chuggin…

> Last time someone had to perform hardware maintenance was at least 3 months ago. If you're small enough (20-30 servers for us) the things mostly run themselves.

If those servers are production servers, I'd say that's not small--you're already scaled to the point where you have hired a whole person (or two) to do server work, no?

I've worked at places where I'm the only technical person. I know enough to set up a Django server, but the reality is I'm not an IT guy, I'm a developer. Paying for a few cloud servers means we don't have to hire anyone to do server maintenance, which gives me an entire person's salary as a server budget before we have to consider whether hiring a person would be better. A lot of companies never reach the size where that becomes a concern.

Of course everybody wants to get big enough that they need a server farm, but as long as you don't vendor-lock yourself, moving from a virtual Linux server on the cloud to a physical server in a colo center is fairly trivial.

My personal website has been running on a single $10/mo Digital Ocean server for 5 years. I think it would be very difficult to beat that price with a colo center and physical hardware. Most businesses are obviously larger than that, but I think it's a bit of an overreach to claim that it's always more expensive to run on the cloud. It just isn't, for many real-life situations.

Post reply on HN