Live data from Hacker News

IBM CEO says there is 'no way' spending on AI data centers will pay off

businessinsider.com

201–210 of 985 posts

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#201

Don’t worry. The same servers will be used for other computing purposes. And maybe that will be profitable. Maybe it will be beneficial to others. But This cycle of investment and loss is a version of distribution of wealth. Some benefit. The banks and loaners always benefit.

That would be true for general purpose servers. But what they want is lots of special purpose AI chips. While is still possible to use that for something else, it's very different from having a generic server farm.

LAN party at the end of the universe?

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#202
post #121

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

It's worse than that in reality, AI chips are on a two year cadence for backwards compatibility (NVIDIA can basically guarantee it, and you probably won't be able to pay real AI devs enough to stick around to make hardware work arounds). So their accounting is optimistic.

5 years is normal-ish depreciation time frame. I know they are gaming GPUs, but the RTX 3090 came out ~ 4.5 years before the RTX 5090. The 5090 has double the performance and 1/3 more memory. The 3090 is still a useful card even after 5 years.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#203
post #37
post #24

As an elder millennial, I just don't know what to say. That a once in a generation allocation of capital should go towards...whatever this all will be, is certainly tragic given current state of the world and its problems. Can't help but see it as the latest in a lifelong series of baffling high stakes decisions of dubious social benefit that have necessarily global consequences.

I'm a younger millennial. I'm always seeing homeless people in my city and it's an issue that I think about on a daily basis. Couldn't we have spent the money on homeless shelters and food and other things? So many people are in poverty, they can't afford basic necessities. The world is shitty. Yes, I know it's all capital from VC firms and investment firms and other private sources , but it's still capital . It shou…

> Couldn't we have spent the money on homeless shelters and food and other things

I suspect this is a much more complicated issue than just giving them food and shelter. Can money even solve it?

How would you allocate money to end obesity, for instance? It's primarily a behavioral issue, a cultural issue

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#204

> In an October letter to the White House's Office of Science and Technology Policy, OpenAI CEO Sam Altman recommended that the US add 100 gigawatts in energy capacity every year. > Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said. And people think the clima…

100GW per year is not going to happen.

The largest plant in the world is the Three Gorges Dam in China at 22GW and it’s off the scales huge. We’re not building the equivalent of four of those every year.

Unless the plan is to power it off Sam Altman’s hot air. That could work. :)

https://en.wikipedia.org/wiki/List_of_largest_power_stations

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#205

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

5 years is long, actually. This is not a GPU thing. It's standard for server hardware.

Because usually it's more efficient for companies to retire the hardware and put in new stuff.

Meanwhile, my 10-15 year old server hardware keeps chugging along just fine in the rack in my garage.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#207

> In an October letter to the White House's Office of Science and Technology Policy, OpenAI CEO Sam Altman recommended that the US add 100 gigawatts in energy capacity every year. > Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said. And people think the clima…

If we moron our way to large-scale nuclear and renewable energy rollout however..

This admin has already killed as much solar and wind and battery as it can.

The only large scale rollout will be payment platforms that will allow you to split your energy costs into "Five easy payments"

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#208

Earlier quoted context omitted.

A typical data centre is $2,500 per year per kW load (including overhead, hvac and so on). If it costs $800,000 to replace the whole rack, then that would pay off in a year if it reduces 320 kW of consumption. Back when we ran servers, we wouldn't assume 100% utilisation but AI workloads do do that; normal server loads would be 10kW per rack and AI is closer to 100. So yeah, it's not hard to imagine power savings of…

Thanks for the numbers! Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle, and the upgrade simply unlocks a higher amount of service revenue per rack?

> Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle,

Power density seems to grow each cycle. But eventually your DC hits power capacity limits, and you have to leave racks empty because there's no power budget.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#209

One thing we saw with the dot-com bust is how certain individuals were able to cash in on the failures, e.g., low cost hardware, domain names, etc. (NB. prices may exceed $2) Perhaps people are already thinking about they can cash in on the floor space and HVAC systems that will be left in the wake of failed "AI" hype

Why do you believe it will fail? Because some companies will not be profitable?

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#210

Earlier quoted context omitted.

5 years is long, actually. This is not a GPU thing. It's standard for server hardware.

Because usually it's more efficient for companies to retire the hardware and put in new stuff. Meanwhile, my 10-15 year old server hardware keeps chugging along just fine in the rack in my garage.

Sample size of 1 though. It's like how I've had hard disks last a decade, but a 100 node Hadoop cluster had 3 die per week after a few years.
Post reply on HN