Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM CEO says there is 'no way' spending on AI data centers will pay off

businessinsider.com

Post reply on HN
Print view View original

171–180 of 985 posts

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#171

Post by taneq »

trollbridge wrote: post #49
zppln wrote: post #45

Earlier quoted context omitted.

I'm a little bit curious about this. Where do all the hardware from the big tech giants usually go once they've upgraded?

I used (relatively) ancient servers (5-10 years in age) because their performance is completely adequate; they just use slightly more power. As a plus it's easy to buy spare parts, and they run on DDR3, so I'm not paying the current "RAM tax". I generally get such a server, max out its RAM, max out its CPUs and put it to work.

Same, the bang for buck on a 5yo server is insane. I got an old Dell a year ago (to replace our 15yo one that finally died) and it was $1200 AUD for a maxed out recently-retired server with 72TB of hard drives and something like 292GB of RAM.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#172

Post by slashdave »

scroot wrote: post #24

As an elder millennial, I just don't know what to say. That a once in a generation allocation of capital should go towards...whatever this all will be, is certainly tragic given current state of the world and its problems. Can't help but see it as the latest in a lifelong series of baffling high stakes decisions of dubious social benefit that have necessarily global consequences.

Well, at least this doesn't involve death and suffering, like the old-fashioned way to jump-start an economy by starting a global war.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#173

Post by q3k »

zppln wrote: post #45
beachy wrote: post #27

Earlier quoted context omitted.

It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.

I'm a little bit curious about this. Where do all the hardware from the big tech giants usually go once they've upgraded?

In-house hyperscaler stuff gets shredded, after every single piece of flash storage gets first drilled through and every hard drive gets bent by a hydraulic press. Then it goes into the usual e-waste recycling stream (ie. gets sent to poor countries where precious metals get extracted by people with a halved life expectancy).

Off-the-shelf enterprise gear has a chance to get a second life through remarketing channels, but much of it also gets shredded due to dumb corporate policies. There are stories of some companies refusing to offload a massive decom onto the second hand market as it would actually cause a crash. :)

It's a very efficient system, you see.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#175

Post by PunchyHamster »

beachy wrote: post #27
Octoth0rpe wrote: post #18

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possib…

It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.

Eh, not exactly. If you don't run CPU at 70%+ the rest of the machine isn't that much more inefficient that model generation or two behind.

It used to be that new server could use half power of the old one at idle but vendors figured out that servers also need proper power management a while ago and it is much better.

Last few gens increase could be summed up to "low % increase in efficiency, with TDP, memory channels and core count increase".

So for loads not CPU bound the savings on newer gen aren't nearly worth it to replace it, and for bulk storage the CPU power usage is even smaller part

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#176

Post by 1vuio0pswjnm7 »

One thing we saw with the dot-com bust is how certain individuals were able to cash in on the failures, e.g., low cost hardware, domain names, etc. (NB. prices may exceed $2)

Perhaps people are already thinking about they can cash in on the floor space and HVAC systems that will be left in the wake of failed "AI" hype

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#177

Post by PunchyHamster »

taneq wrote: post #171
trollbridge wrote: post #49

Earlier quoted context omitted.

I used (relatively) ancient servers (5-10 years in age) because their performance is completely adequate; they just use slightly more power. As a plus it's easy to buy spare parts, and they run on DDR3, so I'm not paying the current "RAM tax". I generally get such a server, max out its RAM, max out its CPUs and put it to work.

Same, the bang for buck on a 5yo server is insane. I got an old Dell a year ago (to replace our 15yo one that finally died) and it was $1200 AUD for a maxed out recently-retired server with 72TB of hard drives and something like 292GB of RAM.

Just not too old. Easy to get into "power usage makes it not worth it" for any use case when it runs 24/7

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#178

Post by throwaway31131 »

milesvp wrote: post #56
skeeter2020 wrote: post #33

The interesting macro view on what's happening is to compare a mature data center operation (specifically a commoditized one) with the utility business. The margins here, and in similar industries with big infra build-out costs (ex: rail) are quite small. Historically the businesses have not done well; I can't really imagine what happens when tech companies who've only ever known huge, juicy margins experience low si…

Worse, is that a lot of these people are acting like Moore's law isn't still in effect. People conflate clock speeds on beefy hardware with moore's law, and act like it's dead, when transistor density rises, and cost per transistor continue to fall at rates similar to what they always have. That means the people racing to build out infrastructure today might just be better off parking that money in a low interest acc…

Cost per transistor is increasing. or flat, if you stay on a legacy node. They pretty much squeezed all the cost out of 28nm that can be had, and it’s the cheapest per transistor.

“based on the graph presented by Milind Shah from Google at the industry tradeshow IEDM, the cost of 100 million transistors normalized to 28nm is actually flat or even increasing.”

https://www.tomshardware.com/tech-industry/manufacturing/chi...

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#179

Post by skywhopper »

arisAlexis wrote: post #66
mbreese wrote: post #62

I would add an addendum to this -- there is no way the announced spending on AI data centers will all come to fruition. I have no doubt that there will be a massive build-out of infrastructure, but it can't reach the levels that have been announced. The power requirements alone will stop that from happening.

[flagged]

lol, CEOs do not do research.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#180

Post by maplethorpe »

irilesscent wrote: post #41
criddell wrote: post #23

> But AGI will require "more technologies than the current LLM path," Krisha said. He proposed fusing hard knowledge with LLMs as a possible future path. And then what? These always read a little like the underpants gnomes business model (1. Collect underpants, 2. ???, 3. Profit). It seems to me that the AGI business models require one company has exclusive access to an AGI model. The reality is that it will likely s…

AGI has value in automation and optimisation which increase profit margins.When AGI is everywhere, then the game is who has the smartest agi, who can offer it cheapest, who can specialise it for my niche etc. Also in this context agi need to run somewhere and IBM stands to benefit from running other peoples models.

> then the game is who has the smartest agi, who can offer it cheapest, who can specialise it for my niche etc.

I always thought the use case for developing AGI was "if it wants to help us, it will invent solutions to all of our problems". But it sounds like you're imagining a future in which companies like Google and OpenAI each have their own AGI, which they somehow enslave and offer to us as a subscription? Or has the definition of AGI shifted?

Post reply on HN