Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AI labs are solving the power problem

newsletter.semianalysis.com

Post reply on HN
Print view View original

61–70 of 270 posts

Re: How AI labs are solving the power problem

#61

Post by discreteevent »

crazygringo wrote: post #29
pdpi wrote: post #7

Part of what bothers me with AI energy consumption isn't just how wasteful it might be from an ecological perspective, it's how brutally inefficient it is compared to the biological "state of the art" — 2000kcal = 8,368 kJ. 8,368 kJ / 86,400 s = 96.9 W. So the benchmark is achieving human-like intelligence on a 100W budget. I'd be very curious to see what can be achieved by AI targeting that power budget.

Is it though? When I ask an LLM research questions, it often answers in 20 seconds what it would take me an entire afternoon to figure out with traditional research. Similarly, I've had times where it wrote me scientific simulation code that would take me 2 days, in around a minute. Obviously I'm cherry-picking the best examples, but I would guess that overall, the energy usage my LLM queries have required is vastly…

For this kind of thinking to work in practice you would need to kill the people that AI makes redundant. This is apart from the fact that right now we are at a choke point where it's much more important to generate less CO2 than it is to write scientific simulation code a little quicker (and most people are using AI for much more unnecessary stuff like marketing)

Re: How AI labs are solving the power problem

#62

Post by FergusArgyll »

beepbooptheory wrote: post #58
dale_glass wrote: post #46

Earlier quoted context omitted.

Not really. I can generate images or get LLM answers in below 15 seconds on mundane hardware. The image generator draws many times faster than any normal person, and the LLM even on my consumer hardware still produces output faster than I can type (and I'm quite good at that), let alone think what to type.

Is "faster" really what we are talking about right now? It could be a lot faster to take a helicopter to work everyday too, versus riding a bike. Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?

There's a billion users. Why do we make massive cities and factories and fields if humans only need 2000 calories a day

Re: How AI labs are solving the power problem

#63

Post by discreteevent »

beepbooptheory wrote: post #58
dale_glass wrote: post #46

Earlier quoted context omitted.

Not really. I can generate images or get LLM answers in below 15 seconds on mundane hardware. The image generator draws many times faster than any normal person, and the LLM even on my consumer hardware still produces output faster than I can type (and I'm quite good at that), let alone think what to type.

Is "faster" really what we are talking about right now? It could be a lot faster to take a helicopter to work everyday too, versus riding a bike. Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?

> It could be a lot faster to take a helicopter to work everyday too, versus riding a bike.

Great analogy.

Re: How AI labs are solving the power problem

#64

Post by miltonlost »

sam-cop-vimes wrote: post #59
thrance wrote: post #27

> Eighteen months ago, Elon Musk shocked the datacenter industry by building a 100,000-GPU cluster in four months. Multiple innovations enabled this incredible achievement, but the energy strategy was the most impressive. xAI entirely bypassed the grid and generated power onsite, using truck-mounted gas turbines and engines. Wow, "truck-mounted gas turbines"? Who else could have mastered such a futuristic tech in so…

I'm no fan of Musk, but you've got to admit it was a clever way to achieve the goal. SemiAnalysis don't do fanboy articles - their research is pretty in-depth. So they are stating it as they see it. The problem ordinary people all over the world have is that governments are allowing this to happen. Maybe if there were stricter regulation it will prevent players such as Musk to come up with such "innovations".

"Getting a permit for 15 turbines after having illegally used 35 turbines that then poisoned the air for the residences around the turbines" is a clever way to achieve the goal? I wouldn't call doing a blatant illegal action "clever", but rather sociopathic.

https://techcrunch.com/2025/07/03/xai-gets-permits-for-15-na...

https://www.politico.com/news/2025/05/06/elon-musk-xai-memph...

Re: How AI labs are solving the power problem

#65

Post by einrealist »

wyldfire wrote: post #42
einrealist wrote: post #33

Earlier quoted context omitted.

The word 'pollution' appears exactly one time in this entire thing, the word 'community' or 'communities' never.

The only way to solve problems like this IMO is to price in the externalities. Tax fossil fuels for the damage they do, in order to reveal their true cost. Then they will never look like the most affordable option, because they're not.

True. The same is true for nuclear energy. I never heard of a nuclear power plant that did not receive substantial subsidies throughout lifetime. Not to forget the nuclear fuel and the efforts required to create it and later to store it.

Re: How AI labs are solving the power problem

#66

Post by jstummbillig »

saghm wrote: post #15
goda90 wrote: post #3

Power problem: solved Natural Gas supply problem: worsened Carbon in the atmosphere problem: worsened

Yeah I guess I'm not the target audience for this because I assumed that "the power problem" was "massive increase in electricity costs for people despite virtually unchanged usage on their part", not "AI companies have to wait too long to be able to start using even more power than they already are": > Nicole Pastore, who has lived in her large stone home near Baltimore’s Johns Hopkins University campus for 18 years…

I understand the instinct but if people seriously think that they are solving any problem by unplugging cell phone chargers, they are simply bad at math. Human time is easily worth more than that, even when working at minimum wage.

That said, it obviously sucks that utility prices are rising for people who can not effortlessly cover that (not to speak of the local pollution, if that's an issue). Maybe some special tax to offset that cost to society towards hyper scalers would be a reasonable way to soften the blow, but I have not done the math.

Re: How AI labs are solving the power problem

#67

Post by quickthrowman »

Aurornis wrote: post #22
Symmetry wrote: post #2

I found Boom's pivot much less confusing after this article.

Boom’s pivot to trying to build turbines for data centers wasn’t surprising when data center deployments started using turbines. Either their CEO saw one of the headlines or their investors forwarded it over and it became their new talking point. What is interesting is how many people saw the Boom announcement and came to believe that Boom was a pioneer of this idea. They’re actually a me-too that won’t have anything…

> What is interesting is how many people saw the Boom announcement and came to believe that Boom was a pioneer of this idea. They’re actually a me-too that won’t have anything ready for a long time, if they can even pull it off at all.

My first thought when seeing that article is “I can buy one of these right now from Siemens or GE, and I could’ve ordered one at any time in the last 50 years.”

Re: How AI labs are solving the power problem

#68

Post by usefulcat »

crazygringo wrote: post #29
pdpi wrote: post #7

Part of what bothers me with AI energy consumption isn't just how wasteful it might be from an ecological perspective, it's how brutally inefficient it is compared to the biological "state of the art" — 2000kcal = 8,368 kJ. 8,368 kJ / 86,400 s = 96.9 W. So the benchmark is achieving human-like intelligence on a 100W budget. I'd be very curious to see what can be achieved by AI targeting that power budget.

Is it though? When I ask an LLM research questions, it often answers in 20 seconds what it would take me an entire afternoon to figure out with traditional research. Similarly, I've had times where it wrote me scientific simulation code that would take me 2 days, in around a minute. Obviously I'm cherry-picking the best examples, but I would guess that overall, the energy usage my LLM queries have required is vastly…

> it often answers in 20 seconds what it would take me an entire afternoon to figure out with traditional research.

In that case I think it would be only fair to also count the energy required for training the LLM.

LLMs are far ahead of humans in terms of the sheer amount of knowledge they can remember, but nowhere close in terms of general intelligence.

Re: How AI labs are solving the power problem

#70

Post by dale_glass »

beepbooptheory wrote: post #58
dale_glass wrote: post #46

Earlier quoted context omitted.

Not really. I can generate images or get LLM answers in below 15 seconds on mundane hardware. The image generator draws many times faster than any normal person, and the LLM even on my consumer hardware still produces output faster than I can type (and I'm quite good at that), let alone think what to type.

Is "faster" really what we are talking about right now? It could be a lot faster to take a helicopter to work everyday too, versus riding a bike. Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?

Speed highly correlates with power efficiency. I believe my hardware maxes out somewhere around 150W. 15 seconds of that isn't much at all.

> Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?

I presume that's mostly training, not inference. But in general anything that serves millions of requests in a small footprint is going to look pretty big.

Post reply on HN