IBM CEO says there is 'no way' spending on AI data centers will pay off

Post by cryptophreak » Wed, Dec 03, 2025, 5:01 AM UTC

nnurmanov wrote: post #434

I agree. Here is my thinking. What if LLM providers will make short answers the default (for example, up to 200 tokens, unless the user explicitly enables “verbose mode”). Add prompt caching and route simple queries to smaller models. Result: a 70%+ reduction in energy consumption without loss of quality. Current cost: 3–5 Wh per request. At ChatGPT scale, this is $50–100 million per year in electricity (at U.S. rate…

Post by getnormality » Wed, Dec 03, 2025, 5:07 AM UTC

stego-tech wrote: post #308

getnormality wrote: post #265

A decade ago, IBM was spending enormous amounts of money to tell me stuff like "cognitive finance is here" in big screen-hogging ads on nytimes.com. They were advertising Watson, vaporware which no one talks about today. Are they bitter that someone else has actually made the AI hype take off?

If anything, the fact they built such tooling might be why they're so sure it won't work. Don't get me wrong, I am incredibly not a fan of their entire product portfolio or business model (only Oracle really beats them out for "most hated enterprise technology company" for me), but these guys have tentacles just as deep into enterprises as Oracle and are coming up dry on the AI front. Their perspective shouldn't be i…

Post by z0mghii » Wed, Dec 03, 2025, 5:16 AM UTC

tharkun__ wrote: post #426

AdrianB1 wrote: post #281

Earlier quoted context omitted.

I thought the same until I calculated that newer hardware consumes a few times less energy and for something running 24x7 that adds up quite a bit (I live in Europe, energy is quite expensive). So my homelab equipment is just 5 years old and it will get replaced in 2-3 years with something even more power efficient.

Where in Europe? Asking coz I just did a quick comparison and it seems to depend but for comparison I have a really old AMD Athlon "e" processor (like literally September 2009 is when it came out according to some quick Google search, tho I probably bought it a few months later than that but still ...) that runs at ~45W TDP. In idle conditions, it typically consumes around 10 to 15 watts (internet wisdom, not kill-a-…

Post by balder1991 » Wed, Dec 03, 2025, 5:20 AM UTC

cryptophreak wrote: post #441

nnurmanov wrote: post #434

I agree. Here is my thinking. What if LLM providers will make short answers the default (for example, up to 200 tokens, unless the user explicitly enables “verbose mode”). Add prompt caching and route simple queries to smaller models. Result: a 70%+ reduction in energy consumption without loss of quality. Current cost: 3–5 Wh per request. At ChatGPT scale, this is $50–100 million per year in electricity (at U.S. rate…

Can you explain why a low-hanging optimization that would reduce costs by 90% without reducing perceived value hasn't been implemented?

Post by physicsguy » Wed, Dec 03, 2025, 5:23 AM UTC

chii wrote: post #425

duped wrote: post #402

Earlier quoted context omitted.

How different is this from rental car companies changing over their fleets? I don't know, this is a genuine question. The cars cost 3-4x as much and last about 2x as far as I know, and the secondary market is still alive.

> the secondary market is still alive. this is the crux. Will these data center cards, if a newer model came out with better efficiency, have a secondary market to sell to? It could be that second hand ai hardware going into consumers' hands is how they offload it without huge losses.