Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI won't use as much electricity as we are told (2024)

johnquigginblog.substack.com

Post reply on HN
Print view View original

81–90 of 124 posts

Re: AI won't use as much electricity as we are told (2024)

#81

Post by catlikesshrimp »

aceazzameen wrote: post #53
taeric wrote: post #21

Earlier quoted context omitted.

Do we use more electricity because of 5G? I confess I'd assume modern phones and repeater networks use less power than older ones. Even at large. I can easily agree that phones that have internet capabilities use more, as a whole, than those that didn't. The infrastructure needs were very different. But, especially if you are comparing to 4G technology, much of that infrastructure already had to distribute content th…

As a data point, I turn 5G off on my phone and get several hours more battery life using 4G. I'm pretty sure the higher bandwidth is consuming more energy, especially since 5G works at shorter distances and probably needs the power to stay connected to cell towers.

HSPA++ Works for HN... But people want 8k TikTok

Re: AI won't use as much electricity as we are told (2024)

#82

Post by danans »

JimDabell wrote: post #70
Tycho wrote: post #47

What’s the energy profile of running inference in a typical ChatGPT prompt compared to: - doing a google search and loading a linked webpage - taking a photo with your smartphone and uploading it to social media for sharing - playing Fortnite for 20 minutes - hosting a Zoom conference with 15 people - sending an email to a hundred colleagues I’d be curious. AI inference is massively centralised, so of course the data…

A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009. AI energy use is negligible compared with other everyday activities. This is a great article on the subject: https://andymasley.substack.com/p/a-cheat-sheet-for-conversa... The same author has published a series of articles that go into a lot of depth when it comes to AI energy and water use: https://andymasley.substack…

> A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009.

That's the number their CEO put out, but AFAIK it is completely unverified (they did not provide any background as to how it was calculated). To believe it is an article of faith at this point.

What is concrete and verifiable are the large deals being struck between AI model providers and energy providers - often to be supplied via fossil fuels.

Re: AI won't use as much electricity as we are told (2024)

#83

Post by prewett »

Majestic121 wrote: post #22
palata wrote: post #9

> But we have been here before. Predictions of this kind have been made ever since the emergence of the Internet I don't think I live in the same world as the author. Ever since the emergence of the Internet, "stuff related to IT" has been using more and more energy. It's like saying "5G won't use as much electricity as we are told! In fact 5G is more efficient than 4G". Yep, except that 5G enables us to use a lot mo…

This is countered in the article. "Yet throughout this period, the actual share of electricity use accounted for by the IT sector has hovered between 1 and 2 per cent, accounting for less than 1 per cent of global greenhouse gas emissions."

But presumably the total use of energy has been going up, so while the relative percentage might be the same, I doubt very much that the absolute quantity of GWhr/year has stayed the same.

Re: AI won't use as much electricity as we are told (2024)

#84

Post by pixl97 »

bob1029 wrote: post #59

I'd be very interested in seeing some kind of aggregated daily demand curve for AI workloads. It seems like a lot of the hyperbolic angles are looking at this as a constant draw of power over time. There is no reason for a GPU inference farm to be ramped up to 100% clock speed when all of its users are in bed. The 5700XT in my computer is probably pulling a mere 8~12W right now since it is just sitting on an idle des…

With the current demand for GPU time it makes no sense to have your units idle. If you're not selling infrence to customers you'll batch out training to whoever will rent the excess time from you.

Maybe in the future there will be idle time.

Re: AI won't use as much electricity as we are told (2024)

#85

Post by xbmcuser »

Personally I don't get some of the power investments by US tech companies. UAE last year signed a project agreement where it would get 1 GWH 24 hours a day from solar and battery project for around $6 billion . Prices for batteries are down a lot further now. So even with adding more battery and solar from more reliability they could build out solar and battery plants with where cost for electricity can be under $0.10/kwh

Re: AI won't use as much electricity as we are told (2024)

#86

Post by belter »

jerf wrote: post #28

It's been a while, but I don't recall any of the dotcom startups making deals with nuclear energy companies to buy out entire nuclear power stations: https://www.npr.org/2024/09/20/nx-s1-5120581/three-mile-isla... And that's just an example, there are many power-related deals of similar magnitude. The companies building out capacity certainly believe that AI is going to use as much power as we are told. We are told t…

"Exposing The Dark Side of America's AI Data Center Explosion" - https://youtu.be/t-8TDOFqkQA

Re: AI won't use as much electricity as we are told (2024)

#87

Post by JimDabell »

danans wrote: post #82
JimDabell wrote: post #70

Earlier quoted context omitted.

A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009. AI energy use is negligible compared with other everyday activities. This is a great article on the subject: https://andymasley.substack.com/p/a-cheat-sheet-for-conversa... The same author has published a series of articles that go into a lot of depth when it comes to AI energy and water use: https://andymasley.substack…

> A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009. That's the number their CEO put out, but AFAIK it is completely unverified (they did not provide any background as to how it was calculated). To believe it is an article of faith at this point. What is concrete and verifiable are the large deals being struck between AI model providers and energy providers - often to b…

> That's the number their CEO put out, but AFAIK it is completely unverified (they did not provide any background as to how it was calculated). To believe it is an article of faith at this point.

Google also puts the median Gemini prompt at 0.24 Wh. The information available from different sources point in the same direction; you don’t have to take Sam Altman’s word for it. 0.3 Wh was the figure that was already pretty dependable before he said that.

> What is concrete and verifiable are the large deals being struck between AI model providers and energy providers - often to be supplied via fossil fuels.

Which is completely irrelevant to this discussion unless you quantify that in Wh per prompt. Vague “deals are being struck!” hand-wringing doesn’t add to the discussion at all. Why are you demanding absolute, unimpeachable rigour when vendors give specific figures, but are comfortable with hand waving when it comes to complaining about energy use?

Re: AI won't use as much electricity as we are told (2024)

#88

Post by quickthrowman »

PTOB wrote: post #23
JohnFen wrote: post #3

> By contrast, the unglamorous and largely disregarded business of making cement accounts for around 7 per cent of global emissions. Oh, that's not a good example of the point they're trying to make. The emissions from concrete are a point of major concern and are frequently discussed. A ton of effort is being put into trying to reduce the problem, and there are widespread calls to reduce the use of the material as m…

Has he considered exactly how much concrete is needed to build a datacenter campus?

A 6” concrete slab (excluding footings) has ~18,500 cubic yards of concrete per million sqft (54 sqft of 6” slab is one cubic yard of concrete)

Re: AI won't use as much electricity as we are told (2024)

#89

Post by bilekas »

timschmidt wrote: post #57
bilekas wrote: post #32

Earlier quoted context omitted.

> Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead Am I missing something or has the need to vast GPU horsepower been solved ? Those requirements were not in DC's before and they're only going up. Whatever way you look at it, there's got to be an increase in power consumption somewhere no ?

GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt. AI is still very ne…

> GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long.

Of course they've been a thing, but for specialised situations, maybe rendering farms or backroom mining centers but it's disingenuous to claim that there's not an exponential growth in gpu useage.

Re: AI won't use as much electricity as we are told (2024)

#90

Post by timschmidt »

philipwhiuk wrote: post #79
timschmidt wrote: post #57

Earlier quoted context omitted.

GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt. AI is still very ne…

> Moore will continue to exponentially decrease costs over time as with all other workloads. There's absolutely no guarantee of this. The continuation of Moore's law is far from certain (NVIDIA think it's dead already).

> NVIDIA think it's dead already

Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too.

Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice.

Until then, they can sit with the group of people who've been vocally wrong about moore's law's end for the last 50 years.

Our chips are still overwhelmingly 2D in design, just a few dozen layers thick but billions of transistors wide. We have quite a ways to go based on a first principles analysis alone. And indeed, that's what chip engineers like Jim Keller say: https://www.youtube.com/watch?v=c01BlUDIlK4

So ask yourself how it benefits Jensen to convince you otherwise.

Post reply on HN