Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM CEO says there is 'no way' spending on AI data centers will pay off

businessinsider.com

Post reply on HN
Print view View original

31–40 of 985 posts

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#31

Post by Octoth0rpe »

mcculley wrote: post #22
Octoth0rpe wrote: post #18

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

But if your competitor is running newer chips that consume less power per operation, aren't you forced to upgrade as well and dispose of the old hardware?

Sure, assuming the power cost reduction or capability increase justifies the expenditure. It's not clear that that will be the case. That's one of the shaky assumptions I'm referring to. It may be that the 2030 nvidia accelerators will save you $2000 in electricity per month per rack, and you can upgrade the whole rack for the low, low price of $800,000! That may not be worth it at all. If it saves you $200k/per rack or unlocks some additional capability that a 2025 accelerator is incapable of and customers are willing to pay for, then that's a different story. There are a ton of assumptions in these scenarios, and his logic doesn't seem to justify the confidence level.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#32

Post by carlCarlCarlCar »

rlpb wrote: post #16
bluGill wrote: post #10

I question depreciation. those gpu's will be obsolete in 5 years, but will the newer be enough better as to be worth replacing them is an open question. cpu's stopped getting exponetially faster 20 years ago, (they are faster but not the jumps the 1990s got)

> those gpu's will be obsolete in 5 years, but will the newer be enough better as to be worth replacing them is an open question Doesn't one follow from the other? If newer GPUs aren't worth an upgrade, then surely the old ones aren't obsolete by definition.

MTBF for data center hardware is short; DCs breeze through GPUs compared to even the hardest of hardcore gamers.

And there is the whole FOMO effect to business purchases; decision makers will worry their models won't be as fast.

Obsolete doesn't mean the reductive notion you have in mind, where theoretically it can still push pixels. Physics will burn them up, and "line go up" will drive demand to replace them.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#33

Post by skeeter2020 »

The interesting macro view on what's happening is to compare a mature data center operation (specifically a commoditized one) with the utility business. The margins here, and in similar industries with big infra build-out costs (ex: rail) are quite small. Historically the businesses have not done well; I can't really imagine what happens when tech companies who've only ever known huge, juicy margins experience low single digit returns on billions of investment.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#34

Post by PrairieFire »

scroot wrote: post #24

As an elder millennial, I just don't know what to say. That a once in a generation allocation of capital should go towards...whatever this all will be, is certainly tragic given current state of the world and its problems. Can't help but see it as the latest in a lifelong series of baffling high stakes decisions of dubious social benefit that have necessarily global consequences.

agree the capital could be put to better use, however I believe the alternative is this capital wouldn't have otherwise been put to work in ways that allow it to leak to the populace at large. for some of the big investors in AI infrastructure, this is cash that was previously and likely would have otherwise been put toward stock buybacks. for many of the big investors pumping cash in, these are funds deploying the wealth of the mega rich, that again, otherwise would have been deployed in other ways that wouldn't leach down to the many that are yielding it via this AI infrastructure boom (datacenter materials, land acquisition, energy infrastructure, building trades, etc, etc)

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#35

Post by skeeter2020 »

dmoy wrote: post #30
Octoth0rpe wrote: post #18

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

5 years is maybe referring to the accounting schedule for depreciation on computer hardware, not the actual useful lifetime of the hardware. It's a little weird to phrase it like that though because you're right it doesn't mean you have to throw it out. Idk if this is some reflection of how IBM handles finance stuff or what. Certainly not all companies throw out hardware the minute they can't claim depreciation on it…

that is how the investments are costed though, so makes sense when we're talking return on investment, so you can compare with alternatives under the same evaluation criteria.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#36

Post by austin-cheney »

Octoth0rpe wrote: post #18

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

It’s not about assumptions on the hardware. It’s about the current demands for computation and expected growth of business needs. Since we have a couple years to measure against it should be extremely straightforward to predict. As such I have no reason to doubt the stated projections.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#37

Post by ayaros »

scroot wrote: post #24

As an elder millennial, I just don't know what to say. That a once in a generation allocation of capital should go towards...whatever this all will be, is certainly tragic given current state of the world and its problems. Can't help but see it as the latest in a lifelong series of baffling high stakes decisions of dubious social benefit that have necessarily global consequences.

I'm a younger millennial. I'm always seeing homeless people in my city and it's an issue that I think about on a daily basis. Couldn't we have spent the money on homeless shelters and food and other things? So many people are in poverty, they can't afford basic necessities. The world is shitty.

Yes, I know it's all capital from VC firms and investment firms and other private sources, but it's still capital. It should be spent on meeting people's basic human needs, not GPU power.

Yeah, the world is shitty, and resources aren't allocated ideally. Must it be so?

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#38

Post by senordevnyc »

ic_fly2 wrote: post #25

IBM might not have a data strategy or AI plan but he isn’t wrong on the inability to generate a profit. A bit of napkin math: NVIDIA claims 0.4J per token for their latest generation 1GW plant with 80% utilisation can therefore produce 6.29 10^16 tokens a year. There are ~10^14 tokens on the internet. ~10^19 tokens have been spoken by humans… so far.

I must be dense, why does this imply AI can't be profitable?

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#39

Post by acdha »

beachy wrote: post #27
Octoth0rpe wrote: post #18

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.

It’s far more extreme: old servers are still okay on I/O, and memory latency, etc. won’t change that dramatically so you can still find productive uses for them. AI workloads are hyper-focused on a single type of work and, unlike most regular servers, a limiting factor in direct competition with other companies.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#40

Post by skeeter2020 »

scarmig wrote: post #20
nabla9 wrote: post #3

Earlier quoted context omitted.

Everyone should read his argument carefully. Ponder them in silence and accept or reject them in based on the strength of the arguments.

His argument follows almost directly, and trivially, from his central premise: a 0% or 1% chance of reaching AGI. Yeah, if you assume technology will stagnate over the next decade and AGI is essentially impossible, these investments will not be profitable. Sam Altman himself wouldn't dispute that. But it's a controversial premise, and one that there's no particular reason to think that the... CEO of IBM would have an…

then it seems like neither Sam Altman (pro) or IBM (proxy con) have credible or even really interesting or insightful evidence, theories ... even suggestions for what's likely to happen? i.e. We should stop listening to all of them?
Post reply on HN