FedEx to close data centers, retire mainframes • Hacker News Board

Re: FedEx to close data centers, retire mainframes

#341

Post by marcosdumay » Tue, Jul 05, 2022, 5:11 PM UTC

jeffbee wrote: post #201

pid-1 wrote: post #130

Earlier quoted context omitted.

That seems to be a common take on HN - cloud is too expensive. I'm curious whether the folks claiming that have any data center ops experience. Because, personally, I'd rather retire than deal with Dell, HP, Cisco, fibers, cooling issues, physical security, hardware failling... And that's just the hardware. Then you still need to pay VMWare for a decent virtualization platform, monitoring tools, etc.... Seriously, no…

1000% this. HN loves to talk about Dropbox. I spent most of my (short, praise God) career at Dropbox diagnosing a fleet of dodgy database servers we bought from HPE. Turns out they were full of little flecks of metal inside. Thousands of em, full of iron filings. You think that kind of thing happens when you are an AWS customer? If you are sophisticated enough to engage an ODM, build your own facilities, and put hvac…

> If you are sophisticated enough to engage an ODM, build your own facilities, and put hvac and electricians on 24-hour payroll, go on-prem. Otherwise, cloud all the way.

I imagine the entire sentiment of the comments is because FedEx is one that really should be sophisticated enough.

Re: FedEx to close data centers, retire mainframes

#342

Post by hinkley » Tue, Jul 05, 2022, 5:11 PM UTC

jes wrote: post #157

cs702 wrote: post #36

Earlier quoted context omitted.

Without a doubt, a future generation of executives will revisit and reverse the decision to rent all information infrastructure, but that will likely be many, many years down the road. In the meantime, the current generation of executives who made this decision will look very smart for saving the company lots of money for a good number of years. And they stand to benefit personally from it. They're doing the rational…

You are right about this. One of the visible signs of an unrecognized (and therefore, unresolved) dilemma is oscillation between two poles. I got this from the late Eli Goldratt and the problem-solving tools he created. One of those tools is the "Evaporating Cloud," which is a way to visualize a dilemma of some kind. I'd suggest that there is an unresolved dilemma here: Should we rent or own data centers? Over a peri…

Often it’s simply the principle of the excluded middle.

If not for the rent seeking behavior of AWS and its cohort, the answer to ”own or rent” would be clear. The answer is “yes”.

Running a data center means you have your own sheep, which means you need shepherds. Shepherds are very useful to have when you have a question about sheep, especially when those questions are about how to manage or use sheep to best effect.

An approachable shepherd can save the rest of the company a lot of money on missteps and bad assumptions, but if you start getting rid of all the sheep, the shepherd will leave too.

We should be using cloud providers for DR, and for regional load balancing. But the company should be maintaining at least one data center of their own, in the same time zones as most of their developers.

I mostly blame Dell and IBM for this. IBM experimented with making server rooms easier to maintain 15-20 years ago and didn’t make it stick. Others ran with some of those ideas. Dell… I don’t know what Dell has done but I know nobody has been writing about it, so from a visibility standpoint they have done nothing.

If/when someone makes it easier (reduced labor) to manage your own servers, the pendulum will swing back.

Re: FedEx to close data centers, retire mainframes

#343

Post by exabrial » Tue, Jul 05, 2022, 5:12 PM UTC

"save"

In the very short term. Data centers are expensive as hell to operate, but so is the cloud. I think colo would have been the way to go.

Re: FedEx to close data centers, retire mainframes

#344

Post by phpisthebest » Tue, Jul 05, 2022, 5:12 PM UTC

menzoic wrote: post #262

kragen wrote: post #19

Ultimately companies that abdicate their informatics operations like this will give their profits to their data-center operators, who will be empowered to charge them whatever price they want. Because what's their BATNA? Migrating from Azure to AWS when Microsoft doesn't want to let them? Renting your information infrastructure is a great way to reduce startup costs, but down the road, that information infrastructure…

>Renting your information infrastructure is a great way to reduce startup costs, but down the road, that information infrastructure runs your company. Trying to outsource it is like trying to outsource upper management. This isn't a one size fits all thing. For a company like FedEx I don't see the advantage of owning their own data centers. They just don't have the scaling challenges that would require that. Data cen…

>>They just don't have the scaling challenges that would require that

Scaling is one of the big cloud advantages. Static Known workloads will almost always be cheaper on Prem than dynamic workloads that need automated scaling

One of the big ways an organization can save money is if they can scale up and down their loads on demand

But if you have a static 24/7 workload almost universally I can build a onprem solution that is be 50% or more cheaper than cloud

Re: FedEx to close data centers, retire mainframes

#345

Post by intrasight » Tue, Jul 05, 2022, 5:14 PM UTC

exabrial wrote: post #343

"save" In the very short term. Data centers are expensive as hell to operate, but so is the cloud. I think colo would have been the way to go.

Most financial analysis are companies are short-term.

But labor costs keep going up and cloud computing costs keep going down.

So I don't see how this is short-termism.

Re: FedEx to close data centers, retire mainframes

#346

Post by hinkley » Tue, Jul 05, 2022, 5:14 PM UTC

filereaper wrote: post #189

Its weird to have companies say they're moving from Mainframes to the Cloud. The Mainframe is a sort of cloud, each one has usually double the spare capacity and you call IBM to unlock it. The cost of running a Mainframe is measured in terms of MIPS based on amount of compute used and there's compute reserved for offloading things like JITs (zAAPS). Essentially a version of cloud compute costs. I can understand if yo…

A lot of the cool tech that Linux and parts of cloud tech are crowing about are just new versions of stuff IBM was doing in the 1990’s. Sometimes I wish I knew some of those people, because I’m sure it would be fascinating to wind them up over a few beers and see what comes out of their mouths.

Re: FedEx to close data centers, retire mainframes

#347

Post by chmod600 » Tue, Jul 05, 2022, 5:15 PM UTC

bob1029 wrote: post #285

filereaper wrote: post #189

Its weird to have companies say they're moving from Mainframes to the Cloud. The Mainframe is a sort of cloud, each one has usually double the spare capacity and you call IBM to unlock it. The cost of running a Mainframe is measured in terms of MIPS based on amount of compute used and there's compute reserved for offloading things like JITs (zAAPS). Essentially a version of cloud compute costs. I can understand if yo…

> Or is everyone here caught up in the "Mainframe old" falsehood. It's probably this. I've spent time trying to advocate for the benefits of these systems, but its like shooting a super soaker into the sun.

Can you elaborate a little? I'll admit I know close to nothing about mainframes.

Re: FedEx to close data centers, retire mainframes

#348

Post by varjag » Tue, Jul 05, 2022, 5:17 PM UTC

nix23 wrote: post #259

yobbo wrote: post #242

Earlier quoted context omitted.

Did they pass typical memory/reliability tests and so on?

Maybe in the first day's they survive it, but the flakes are 99% from the fans/bearings, that's why you test servers at max load for at least 1 week and HD's for 2-4 weeks. But i don't think they made even a initial load-/stresstest. Unpack it, trow it into the rack, no checking of internal plug's just nothing...pretty sure about that.

Metal chips is squarely in the long tail of failure modes that you can't really anticipate (but of course really easy to be smug about in hindsight). It is also extremely unlikely the bearings, most likely these are from chassis frames assy not cleaned up properly.

Re: FedEx to close data centers, retire mainframes

#349

Post by marcosdumay » Tue, Jul 05, 2022, 5:17 PM UTC

dehrmann wrote: post #317

fires10 wrote: post #245

Earlier quoted context omitted.

You generally can not go from city water/sewage to on-prem for legal reasons. However, many are going on-prem with solar. I am going almost completely off-grid for cost and SLA requirements.

What's interesting is people seem really excited about installing rooftop solar, but I see fewer companies building collectors in places with cheap land and ample sun. That tells me part of the selling point of rooftop solar is it feels good, and the economics might not actually work.

> but I see fewer companies building collectors in places with cheap land and ample sun

You mean there are fewer companies making large investments than smaller ones?

Re: FedEx to close data centers, retire mainframes

#350

Post by shuntress » Tue, Jul 05, 2022, 5:18 PM UTC

menzoic wrote: post #262

kragen wrote: post #19

Ultimately companies that abdicate their informatics operations like this will give their profits to their data-center operators, who will be empowered to charge them whatever price they want. Because what's their BATNA? Migrating from Azure to AWS when Microsoft doesn't want to let them? Renting your information infrastructure is a great way to reduce startup costs, but down the road, that information infrastructure…

>Renting your information infrastructure is a great way to reduce startup costs, but down the road, that information infrastructure runs your company. Trying to outsource it is like trying to outsource upper management. This isn't a one size fits all thing. For a company like FedEx I don't see the advantage of owning their own data centers. They just don't have the scaling challenges that would require that. Data cen…

> I think its more fair to so that FedEx building their own datacenters is like building their own roads to deliver packages on.

Your metaphor is a little bit off. This is more comparable to FedEx renting vs owning their fleet vehicles.

Obviously they aren't going to be setting up foundries to to scratch-build engines and network switches. But it probably makes as much sense for them to own the buildings & computers running their logistics software as it does for them to own the buildings and vehicles running their logistics hardware.

As an aside, I'm sure FedEx would LOVE to get customers locked in to private FedEx roads with private FedEx addresses that no one else is allowed to deliver too. Thankfully, our system of public infrastructure is robust enough to make this infeasible.