Live data from Hacker News

Estimating AI energy use

spectrum.ieee.org

11–20 of 104 posts

Re: Estimating AI energy use

#11
My thoughts.

Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute.

The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit the limits. There will be some advancements, but it's clear we're already at the flat part of the current s-curve.

There's probably some interesting new architectures already in the works either from postdocs or in tiny startups that will become the base of the next curve in the next 18 months. If so, one or more may be able to take advantage of the current overbuild in data centers.

However, compute has an expiration date like old milk. It won't physically expire but the potential economic potential decreases as tech increases. But if the timing is right, there is going to be a huge opportunity for the next early adopters.

So what's next?

Re: Estimating AI energy use

#12
post #5
post #3

If I'm interpreting this right it's estimating that ChatGPT's daily energy usage is enough to charge just 14,000 electric vehicles - and that's to serve in the order of ~100 million daily users.

> We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence. what do you think the odds of this being accurate are? zero?

[deleted]

Re: Estimating AI energy use

#13
post #3

If I'm interpreting this right it's estimating that ChatGPT's daily energy usage is enough to charge just 14,000 electric vehicles - and that's to serve in the order of ~100 million daily users.

I was about to post this exact thing. Seems ... low? And it will only get more efficient going forward. I don't get why this is supposed to be a big deal for infrastructure since there's definitely way more than 14,000 EVs out there and we are doing well.

[deleted]

Re: Estimating AI energy use

#14
One thing it's doing is jacking up electricity rates for US States that are part of the [PJM Interconnection grid](https://en.wikipedia.org/wiki/PJM_Interconnection). It's a capacity auction price that is used to guarantee standby availability and it is [up significantly](https://www.toledochamber.com/blog/watts-up-why-ohios-electr...) at $270.43 per MW/day, which is far above prior years (~$29–58/MW/day) and this is translating to significantly higher consumer prices.

Re: Estimating AI energy use

#15
post #5
post #3

If I'm interpreting this right it's estimating that ChatGPT's daily energy usage is enough to charge just 14,000 electric vehicles - and that's to serve in the order of ~100 million daily users.

> We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence. what do you think the odds of this being accurate are? zero?

Why would you assume that? It’s in line with estimates that were around before he posted that article and it’s higher than Gemini. It’s a pretty unsurprising number.

Re: Estimating AI energy use

#16
post #11

My thoughts. Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute. The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit…

It's a line (remindme! 5 years)

Re: Estimating AI energy use

#17
post #11

My thoughts. Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute. The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit…

This is one possibility I'm assuming as well. It largely depends on how long this bubble lasts. At the current growth rate it will be unsustainable before many very large DCs can be built so it's possible the impact may not be as severe as the telecom crash.

Another possibility is that new breakthroughs significantly reduce computational needs, efficiency significantly improves, or some similar improvements that reduce DC demand.

Re: Estimating AI energy use

#18
post #14

One thing it's doing is jacking up electricity rates for US States that are part of the [PJM Interconnection grid]( https://en.wikipedia.org/wiki/PJM_Interconnection ). It's a capacity auction price that is used to guarantee standby availability and it is [up significantly]( https://www.toledochamber.com/blog/watts-up-why-ohios-electr... ) at $270.43 per MW/day, which is far above prior years (~$29–58/MW/day) and thi…

Why are consumers paying for electricity used by server farms? Why can't the electricity companies charge the server farms instead?

Where I live, the utility company bills you at a higher rate if you use more electricity.

Re: Estimating AI energy use

#20
post #7
post #5

Earlier quoted context omitted.

> We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence. what do you think the odds of this being accurate are? zero?

Hard to say. Sam wrote that on June 10th this year: https://blog.samaltman.com/the-gentle-singularity GPT-5 came out on 7th August. Assuming the 0.34 value was accurate in the GPT-4o era, is the number today still in the same ballpark or is it wildly different?

I would bet that it's far lower now. Inference is expensive we've made extraordinary efficiency gains through techniques like distillation. That said, GPT-5 is a reasoning model, and those are notorious for high token burn. So who knows, it could be a wash. But selective pressures to optimize for scale/growth/revenue/independence from MSFT/etc makes me think that OpenAI is chasing those watt-hours pretty doggedly. So 0.34 is probably high...

...but then Sora came out.

Post reply on HN