Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimating AI energy use

spectrum.ieee.org

Post reply on HN
Print view View original

81–90 of 104 posts

Re: Estimating AI energy use

#81

Post by kingstnap »

KPGv2 wrote: post #40
geuis wrote: post #11

My thoughts. Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute. The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit…

> There's probably some interesting new architectures already in the works either from postdocs or in tiny startups It is not clear to me why we will have a breakthrough after virtually no movement on this front for decades. Backpropagation is literally 1960s technology.

Your comment sort of implies that all this is some super standardized flow that is well studied and highly optimized but in my experience all this ML stuff is closer to the edge of broken than some kind of local maximum.

There is an ungodly number of engineering decisions that go into making ML work and any number of stupid things are all over the place that cause stuff to fail.

Like something stupid like your normalization was bad or your mix of data was bad or your learning rates were bad or you have some precision issues or your model has a bad init or some architectural problems cause poor training or straight up there are tons of bugs somewhere like your batching was doing something silly or there is some numerically unstable division or sqrt or somewhere etc etc.

At scale with stupid issues like hardware faults I imagine this only gets exponentially worse.

And then on product sides of integrating stuff more bugs sneak in like so many labs were releasing so many open source LLMs with broken and incorrectly configured chat templates that massively tanked performance.

Or they set up some parmeters in sampling wrong and stuff gets stuck in loops or hallucinates tons or something.

In his 2025 hotchips keynote Noam Shazeer (GDM VP) even says that you need hardware determinism because there are just so many bugs in ML experiments that you need to be able to tweak and test things.

Also there are just so many obvious issues with the way everything works conventionally in GPT2 style like with softmax causing attention sinks at punctuation and creating dispersion over longer sequences because of low sharpness and the whole previllaged basis thing making it so common information takes up a lot of model capacity.

Re: Estimating AI energy use

#82

Post by notjulianjaynes »

olejorgenb wrote: post #80
qnleigh wrote: post #31

For reference, global energy consumption is about 180,000 TWh[1]. So while the numbers in this article are large, they're not a significant fraction of the total. Traveling and buying things are probably a much bigger part of your carbon footprint. For example: - 25 LLM queries: ~8.5 Wh - driving one mile: ~250-1000 Wh - one glass bottle: ~1000 Wh [2] - a new laptop: ~600,000 Wh [3] - round-trip flight from LA to Tok…

How did you get "one glass bottle: ~1000 Wh"? [2] does not cite energy use, only CO2 emissions

Not OP but the EPA has a calculator for estimating emissions to energy consumption. https://www.epa.gov/energy/greenhouse-gas-equivalencies-calc...

Re: Estimating AI energy use

#83

Post by AlecSchueler »

phil21 wrote: post #73
AlecSchueler wrote: post #69

Earlier quoted context omitted.

I think the point is that we all need to use less energy, we need to avoid flights from LA to Tokyo where possible, not using the energy use as an excuse to use even more energy.

> we all need to use less energy We need cheaper and cleaner forms of energy. More efficient uses of energy. I do not agree that we "all" need to use less energy overall. Energy use tracks wealth pretty closely, and manufacturing/creating things tends to be energy intensive. The more cheap clean energy we make available, the more novel uses will be found for it.

> We need cheaper and cleaner forms of energy. More efficient uses of energy.

Yep that's the dream, but it's not what I have coming out of my wall right now.

> Energy use tracks wealth pretty closely,

I'm guessing the majority of users on this site are in the 1% globally so it seems reasonable to consider what's produced/manufactured for us and what services like these that we're using

> The more cheap clean energy we make available, the more novel uses will be found for it.

That will be a brilliant future but it's not the reality today.

Re: Estimating AI energy use

#84

Post by simonw »

deaux wrote: post #62
JimDabell wrote: post #15

Earlier quoted context omitted.

Why would you assume that? It’s in line with estimates that were around before he posted that article and it’s higher than Gemini. It’s a pretty unsurprising number.

Back in the 1980s, I'm sure Philip Morris' claimed similar numbers on cigarettes (not) causing cancer as R.J. Reynolds did. I also wouldn't be surprised if Aramco and Rosneft gave similar estimates on global warming and oil's role in it.

"Does smoking cause cancer?" and "Does burning fossil fuels cause global warming?" are a different category from "How much energy does it take to run a prompt?"

The prompt energy question is something that companies can both actively measure and need to actively measure in order to plan their budgets. It's their job to know the answer.

Those other questions, sadly, fall more into the category of it's their job not to know the answer.

Re: Estimating AI energy use

#86

Post by kingstnap »

kingstnap wrote: post #81
KPGv2 wrote: post #40

Earlier quoted context omitted.

> There's probably some interesting new architectures already in the works either from postdocs or in tiny startups It is not clear to me why we will have a breakthrough after virtually no movement on this front for decades. Backpropagation is literally 1960s technology.

Your comment sort of implies that all this is some super standardized flow that is well studied and highly optimized but in my experience all this ML stuff is closer to the edge of broken than some kind of local maximum. There is an ungodly number of engineering decisions that go into making ML work and any number of stupid things are all over the place that cause stuff to fail. Like something stupid like your normal…

I'd like to add to this that in the recent Y combinator podcast with Anthropic head of pretraining the bugs issue is brought up as a major issue[1].

It is so easy to have good ideas broken by random bugs everywhere...

[1] https://youtu.be/YFeb3yAxtjE?t=2919

Re: Estimating AI energy use

#87

Post by epistasis »

phil21 wrote: post #78
epistasis wrote: post #41

Earlier quoted context omitted.

Nobody is actively building out nuclear power. Microsoft is turning on a recently decommissioned facility. New nuclear is too expensive to make sense. At most there are small investments in flash-in-the-pan startups that are failing to deliver plans for small modular reactors. The real build out that will happen is solar/wind with tons of batteries, which is so commonplace that it doesn't even make the news. Those ca…

Plenty of bets being placed on nuclear, but they are moonshot style bets. From where I'm standing, the immediate capital seems to be being deployed at smaller-scale (2-5MW) natural gas turbines co-located on site with the load. I haven't heard a whole lot of battery deployments at the same scale. Of course turbines are now out at 2029 or something for delivery. Only marginally at the edge of this space these days tho…

As far as the grid goes, there's a tiny bit of gas additions, but it's mostly solar, battery, and wind:

https://www.eia.gov/todayinenergy/detail.php?id=65964

Of course, remember that nameplate capacities from different technologies should be corrected for capacity factor, which is roughly 60% for gas, 40% for wind, and 25% for solar, but pre-correction EIA expects

    solar: 33.3GW
    battery: 18.3GW
    wind: 7.7GW
    gas: 4.7GW
And then there's an expected retirement of 1.6GW of old gas this year.

I'm pretty disconnected from the data center folks, but in general the current political environment is highly disfavorable to solar and batteries, and using them too much could have lots of political blowback that is very expensive.

Of course, small gas also has the benefit that the operating costs are spread over the lifetime, rather than being an up-front cost. So even if solar+batteries is cheaper than gas over the lifetime of the system, gas may seem more expedient if you don't want a lot of capital on the books.

Re: Estimating AI energy use

#88

Post by epistasis »

lovecg wrote: post #51
epistasis wrote: post #41

Earlier quoted context omitted.

Nobody is actively building out nuclear power. Microsoft is turning on a recently decommissioned facility. New nuclear is too expensive to make sense. At most there are small investments in flash-in-the-pan startups that are failing to deliver plans for small modular reactors. The real build out that will happen is solar/wind with tons of batteries, which is so commonplace that it doesn't even make the news. Those ca…

> The real build out that will happen is solar/wind with tons of batteries That actually sounds awesome, is there a downside I’m not seeing?

If you're a utility, you may not like that solar and batteries are driving down electricity costs and reducing grid expenses. But even with the thumbs against the scale, we are seeing the most nameplate deployment (see caveats in my parallel reply) in decades, and will likely set a record, because of solar, batteries, and wind in that order:

https://www.eia.gov/todayinenergy/detail.php?id=65964

Re: Estimating AI energy use

#90

Post by pixl97 »

AlecSchueler wrote: post #69
exitb wrote: post #57

Earlier quoted context omitted.

These don't have to be dependent to be meaningful.

I think the point is that we all need to use less energy, we need to avoid flights from LA to Tokyo where possible, not using the energy use as an excuse to use even more energy.

Do we need to use less energy, or do we need to use less fossil fuel based energy?
Post reply on HN