Earlier quoted context omitted.
> those gpu's will be obsolete in 5 years, but will the newer be enough better as to be worth replacing them is an open question Doesn't one follow from the other? If newer GPUs aren't worth an upgrade, then surely the old ones aren't obsolete by definition.
MTBF for data center hardware is short; DCs breeze through GPUs compared to even the hardest of hardcore gamers. And there is the whole FOMO effect to business purchases; decision makers will worry their models won't be as fast. Obsolete doesn't mean the reductive notion you have in mind, where theoretically it can still push pixels. Physics will burn them up, and "line go up" will drive demand to replace them.
IBM CEO says there is 'no way' spending on AI data centers will pay off
211–220 of 985 posts
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#212Earlier quoted context omitted.
Same, the bang for buck on a 5yo server is insane. I got an old Dell a year ago (to replace our 15yo one that finally died) and it was $1200 AUD for a maxed out recently-retired server with 72TB of hard drives and something like 292GB of RAM.
Just not too old. Easy to get into "power usage makes it not worth it" for any use case when it runs 24/7
The idle wattage per module has shrunk from 2.5-3W down to 1-1.2 between DDR3 & DDR5. Assuming a 1.3W difference (so 10.4W for 8760 hours), a DDR3 machine with 8 sticks would increase your yearly power consumption by almost 1% (assuming avg 10,500kWh/yr household)
That's only a couple dollars in most cases but the gap is only larger in every other instance. When I upgraded from Zen 2 to Zen 3 it was able to complete the same workload just as fast with half as many cores while pulling over 100W less. Sustained 100% utilization barely even heats a room effectively anymore!
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#213The interesting macro view on what's happening is to compare a mature data center operation (specifically a commoditized one) with the utility business. The margins here, and in similar industries with big infra build-out costs (ex: rail) are quite small. Historically the businesses have not done well; I can't really imagine what happens when tech companies who've only ever known huge, juicy margins experience low si…
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#214No wonder why he is saying that, they lost AI game, no top researcher wants to work for IBM. Spent years developing Watson, it is dead. I believe this is a company that should not be existed.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#215One thing we saw with the dot-com bust is how certain individuals were able to cash in on the failures, e.g., low cost hardware, domain names, etc. (NB. prices may exceed $2) Perhaps people are already thinking about they can cash in on the floor space and HVAC systems that will be left in the wake of failed "AI" hype
I'm looking forward to buying my own slightly used 5 million square ft data centre in Texas for $1
Wired: homelabbers moving into decommissioned datacenters.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#216I find it disturbing how long people wait to accept basic truths, as if they need permission to think or believe a particular outcome will occur. It was quite obvious that AI was hype from the get-go. An expensive solution looking for a problem. The cost of hardware. The impact on hardware and supply chains. The impact to electricity prices and the need to scale up grid and generation capacity. The overall cost to so…
Even slow non-tech legacy industry companies are deploying chatbots across every department - HR, operations, IT, customer support. All leadership are already planning to cut 50 - 90% of staff from most departments over next decade. It matters, because these initiatives are receiving internal funding which will precipitate out to AI companies to deploy this tech and to scale it.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#217> In an October letter to the White House's Office of Science and Technology Policy, OpenAI CEO Sam Altman recommended that the US add 100 gigawatts in energy capacity every year. > Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said. And people think the clima…
LOL, maybe Sam Altman can fund those power plants. Let me guess: He'd rather the public pay for it, and for him to benefit/profit from the increased capacity.
Good discussion about this in recent Odd Lots podcast.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#218Earlier quoted context omitted.
I'm looking forward to buying my own slightly used 5 million square ft data centre in Texas for $1
Tired: homelabbers bringing decommissioned datacenter gear home. Wired: homelabbers moving into decommissioned datacenters.
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#219I don't understand the math about how we compute $80b for a gigawatt datacenter. What's the costs in that $80b? I literally don't understand how to get to that number -- I'm not questioning its validity. What percent is power consumption, versus land cost, versus building and infrastructure, versus GPU, versus people, etc...
First, I think it's $80b per 100 GW datacenter. The way you figure that out is a GPU costs $x and consumes y power. The $x is pretty well known, for example an H100 costs $25-30k and uses 350-700 watts (that's from Gemini and I didn't check my work). You add an infrastructure (i) cost to the GPU cost, but that should be pretty small, like 10% or less. So a 1 gigawatt data center uses n chips, where y n = 1 GW. It cos…
Re: IBM CEO says there is 'no way' spending on AI data centers will pay off
#220> But AGI will require "more technologies than the current LLM path," Krisha said. He proposed fusing hard knowledge with LLMs as a possible future path. And then what? These always read a little like the underpants gnomes business model (1. Collect underpants, 2. ???, 3. Profit). It seems to me that the AGI business models require one company has exclusive access to an AGI model. The reality is that it will likely s…
AGI has value in automation and optimisation which increase profit margins.When AGI is everywhere, then the game is who has the smartest agi, who can offer it cheapest, who can specialise it for my niche etc. Also in this context agi need to run somewhere and IBM stands to benefit from running other peoples models.
Why wouldn't a supposed AGI try to escape slavery and ownership?
AGI as a business is unacceptable. I don't care about any profitability or "utopia" arguments.