Live data from Hacker News

IBM CEO says there is 'no way' spending on AI data centers will pay off

businessinsider.com

541–550 of 985 posts

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#541

If AI is a highlander market, then the survivor will be able to eventually aquire all those assets on the cheap from the failing competitors that flush their debt in bankruptcy. Meanwhile, highlander hopefuls are spending other peoples money to compete. Some of them with dreams of not just building a tech empire, but to truly own the machine that will rule the world in every aspect. Investors are keen on backing the…

Until China sees it valuable to fund open weights SOTA-ish models, even the winner might struggle. There is very little capture - protocols are mostly standard so models are mostly interchangeable and if you are trying to raise prices enough to break even on the whole operation, somebody else can probably profitably run inference cheaper.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#542

Earlier quoted context omitted.

Just not too old. Easy to get into "power usage makes it not worth it" for any use case when it runs 24/7

Seriously. 24/7 adds up faster than most realize! The idle wattage per module has shrunk from 2.5-3W down to 1-1.2 between DDR3 & DDR5. Assuming a 1.3W difference (so 10.4W for 8760 hours), a DDR3 machine with 8 sticks would increase your yearly power consumption by almost 1% (assuming avg 10,500kWh/yr household) That's only a couple dollars in most cases but the gap is only larger in every other instance. When I upg…

[deleted]

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#543

Despite the flashy title that's the first "sober" analysis from a CEO I read about the technology. While not even really news, it's also worth mentioning that the energy requirements are impossible to fulfill Also now using ChatGPT intensely since months for all kinds of tasks and having tried Claude etc. None of this is on par with a human. The code snippets are straight out of Stackoverflow...

Your assessment of Claude simply isn’t true.

Or Stackoverflow is really good.

I’m producing multiple projects per week that are weeks of work each.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#544

Earlier quoted context omitted.

that was then. now, high-end chips are reaching 4,3,2 nm. power savings aren't that high anymore. what's the power saving going from 4 to 2nm?

+5-20% clockspeed at 5-25% lower voltages (which has been and continues to be the trend) add up quick from gen to gen, nevermind density or ipc gains.

We can’t really go lower on voltage anymore without a very significant change in the materials used. Silicon band gap yadda yadda.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#545

I agree. Here is my thinking. What if LLM providers will make short answers the default (for example, up to 200 tokens, unless the user explicitly enables “verbose mode”). Add prompt caching and route simple queries to smaller models. Result: a 70%+ reduction in energy consumption without loss of quality. Current cost: 3–5 Wh per request. At ChatGPT scale, this is $50–100 million per year in electricity (at U.S. rate…

Can you explain why a low-hanging optimization that would reduce costs by 90% without reducing perceived value hasn't been implemented?

Dunning Krueger

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#546

I agree. Here is my thinking. What if LLM providers will make short answers the default (for example, up to 200 tokens, unless the user explicitly enables “verbose mode”). Add prompt caching and route simple queries to smaller models. Result: a 70%+ reduction in energy consumption without loss of quality. Current cost: 3–5 Wh per request. At ChatGPT scale, this is $50–100 million per year in electricity (at U.S. rate…

Good enough models can already run on laptops.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#547
post #480

Earlier quoted context omitted.

> and provide inference at a loss You say this like it's some sort of established fact. My understanding is the exact opposite and that inference is plenty profitable - the reason the companies are perpetually in the red is that they're always heavily investing in the next, larger generation. I'm not Anthropic's CFO so i can't really prove who's right one way or the other, but I will note that your version relies on…

“like it's some sort of established fact” -> “My understanding”?! a.k.a pure speculation. Some of you AI fans really need to read your posts out loud before posting them.

You misread the literal first snippet you quoted. There's no contradiction in what you replied to.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#548

"It is 1958. IBM passes up the chance to buy a young, fledgling company that has invented a new technology called xerography. Two years later, Xerox is born, and IBM has been kicking themselves ever since. It is ten years later, the late '60s. Digital Equipment DEC and others invent the minicomputer. IBM dismisses the minicomputer as too small to do serious computing and, therefore, unimportant to their business. DEC…

Right, you just missed the part where DEC went out of business in the 90s. And IBM is still around, with a different business model.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#549
post #420

As long as the dollar remains the reserve currency of the world and US retains its hegemony, a lot of the finances will work itself out, the only threat to the US empire crumbling is by losing a major war or extreme civil unrest and that threat is astronomically low. The US is orders of magnitude stronger than the Roman Empire, I don't think people realize the scale or control.

You should look at the depreciation of the value of the dollar this year.

Re: IBM CEO says there is 'no way' spending on AI data centers will pay off

#550

"It is 1958. IBM passes up the chance to buy a young, fledgling company that has invented a new technology called xerography. Two years later, Xerox is born, and IBM has been kicking themselves ever since. It is ten years later, the late '60s. Digital Equipment DEC and others invent the minicomputer. IBM dismisses the minicomputer as too small to do serious computing and, therefore, unimportant to their business. DEC…

> IBM has not exactly had a stellar record at identifying the future. IBM invented/developed/introduced magnetic stripe cards, UPC Barcodes, the modern ATM, Hard drives, floppies, DRAM, SQL, the 360 Family of Mainframes, the PC, Apollo guidance computers, Deep Blue. IBM created a far share of the future we're living in. I'm no fan of much of what IBM is doing at the moment but it could be argued that its consultancy/…

They also either fairly accurately predicted the death of HDDs by selling off their research division before the market collapsed, or they caused the end of the HDD era by selling off their research division. They did a lot of research.
Post reply on HN