How long can ai gpus stretch? Optmistic 10 years and we're still looking at 400b+ profit to cover interests. The factor in silicon is closer to tulips than rail or fiber in terms of depreciated assets.
IBM CEO says there is 'no way' spending on AI data centers will pay off
21–30 of 985 posts
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#22> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…
But if your competitor is running newer chips that consume less power per operation, aren't you forced to upgrade as well and dispose of the old hardware?
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#23> But AGI will require "more technologies than the current LLM path," Krisha said. He proposed fusing hard knowledge with LLMs as a possible future path.
And then what? These always read a little like the underpants gnomes business model (1. Collect underpants, 2. ???, 3. Profit). It seems to me that the AGI business models require one company has exclusive access to an AGI model. The reality is that it will likely spread rapidly and broadly.
If AGI is everywhere, what's step 2? It seems like everything AGI generated will have a value of near zero.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#24As an elder millennial, I just don't know what to say. That a once in a generation allocation of capital should go towards...whatever this all will be, is certainly tragic given current state of the world and its problems. Can't help but see it as the latest in a lifelong series of baffling high stakes decisions of dubious social benefit that have necessarily global consequences.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#25IBM might not have a data strategy or AI plan but he isn’t wrong on the inability to generate a profit.
A bit of napkin math: NVIDIA claims 0.4J per token for their latest generation 1GW plant with 80% utilisation can therefore produce 6.29 10^16 tokens a year.
There are ~10^14 tokens on the internet. ~10^19 tokens have been spoken by humans… so far.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#26I question depreciation. those gpu's will be obsolete in 5 years, but will the newer be enough better as to be worth replacing them is an open question. cpu's stopped getting exponetially faster 20 years ago, (they are faster but not the jumps the 1990s got)
I recently compared performance per dollar for CPUs and GPUs on benchmarks for GPUs today vs 10 years ago, and suprisingly, CPUs had much bigger gains. Until I saw that for myself, I thought exactly the same thing as you.
It seems shocking given that all the hype is around GPUs.
This probably wouldn't be true for AI specific workloads because one of the other things that happened there in the last 10 years was optimising specifically for math with lower size floats.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#27> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…
It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#28I question depreciation. those gpu's will be obsolete in 5 years, but will the newer be enough better as to be worth replacing them is an open question. cpu's stopped getting exponetially faster 20 years ago, (they are faster but not the jumps the 1990s got)
I think real issue is current costs / demand = Nvidia gouging GPU price that costs for hardware:power consumption is 70:20 instead of 50:40 (10 for rest of datacenter). Reality is gpus are serendipidous path dependent locked from gaming -> mining. TPUs are more power efficient, if bubble pops and demand for compute goes down, Nvidia + TMSC will still be around, but nexgen AI first bespoke hardware premium will revert towards mean and we're looking at 50% less expensive hardware (no AI race scarcity tax, i.e. 75% Nvidia margins) that use 20% less power / opex. All of a sudden existing data centers becomes not profitable stranded assets even if they can be stretched past 5 years.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#29IBM might not have a data strategy or AI plan but he isn’t wrong on the inability to generate a profit. A bit of napkin math: NVIDIA claims 0.4J per token for their latest generation 1GW plant with 80% utilisation can therefore produce 6.29 10^16 tokens a year. There are ~10^14 tokens on the internet. ~10^19 tokens have been spoken by humans… so far.
> ~10^14 tokens on the internet
Does that include image tokens? My bet is with image tokens you are off by at least 5 orders of magnitude for both.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#30> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…
5 years is maybe referring to the accounting schedule for depreciation on computer hardware, not the actual useful lifetime of the hardware.
It's a little weird to phrase it like that though because you're right it doesn't mean you have to throw it out. Idk if this is some reflection of how IBM handles finance stuff or what. Certainly not all companies throw out hardware the minute they can't claim depreciation on it. But I don't know the numbers.
Anyways, 5 years is an infection point on numbers. Before 5 years you get depreciation to offset some cost of running. After 5 years, you do not, so the math does change.