Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI won't use as much electricity as we are told (2024)

johnquigginblog.substack.com

Post reply on HN
Print view View original

51–60 of 124 posts

Re: AI won't use as much electricity as we are told (2024)

#51

Post by js8 »

I found this video https://youtu.be/IQvREfKsVXM interesting, especially because it mentions couple of AI studies/papers that argue in favor of much smaller (and more efficient) models. (And I have never heard of them.)

I suspect that yes, for AGI much smaller models will eventually prove to be sufficient. I think in 20 years everyone will have an AI agent in their phone, busily exchanging helpful information with other AI agents of people who you trust.

I think the biggest problem with tech companies is they effectively enclosed and privatized the social graph. I think it should be public, i.e. one shouldn't have to go through a 3rd party to make an inquiry for how much someone trusts a given source of information, or where the given piece of information originated. (There is more to be written about that topic but it's only marginally related to AI.)

Re: AI won't use as much electricity as we are told (2024)

#52

Post by patapong »

afavour wrote: post #33

That's an awful amount of certainty for something that isn't backed by very much certainty at all. Just "previous claims about inefficiency in tech have ended up being incorrect". As a counterpoint: look at crypto. The amount of power used by cryptocurrency has _not_ gone down, in fact it's increased.

While I don't disagree with your overall point, I don't think crypto is a good counterpoint here. Crypto is conditioned on using more and more energy to secure the network. As the value increases, more mining hardware can be thrown at it, which increases security adn thus value - there is no upper bound.

AI on the other hand aims at both increased quality but also reduced energy consumption. While there are certainly developments that favour the latter at the cost of the latter (e.g. reasoning models), there are also indications that companies are finding ways to make the models more efficient while maintaining quality. For example, the moves from GPT-4 -> GPT-4-turbo and 4o -> 5 were speculated to be in the service of efficiency. Hopefully the market forces that make computing cheaper and more energy effective will also push AI to become more energy effective over time.

Re: AI won't use as much electricity as we are told (2024)

#53

Post by aceazzameen »

taeric wrote: post #21
palata wrote: post #9

> But we have been here before. Predictions of this kind have been made ever since the emergence of the Internet I don't think I live in the same world as the author. Ever since the emergence of the Internet, "stuff related to IT" has been using more and more energy. It's like saying "5G won't use as much electricity as we are told! In fact 5G is more efficient than 4G". Yep, except that 5G enables us to use a lot mo…

Do we use more electricity because of 5G? I confess I'd assume modern phones and repeater networks use less power than older ones. Even at large. I can easily agree that phones that have internet capabilities use more, as a whole, than those that didn't. The infrastructure needs were very different. But, especially if you are comparing to 4G technology, much of that infrastructure already had to distribute content th…

As a data point, I turn 5G off on my phone and get several hours more battery life using 4G. I'm pretty sure the higher bandwidth is consuming more energy, especially since 5G works at shorter distances and probably needs the power to stay connected to cell towers.

Re: AI won't use as much electricity as we are told (2024)

#54

Post by everdrive »

Arnt wrote: post #29
palata wrote: post #9

> But we have been here before. Predictions of this kind have been made ever since the emergence of the Internet I don't think I live in the same world as the author. Ever since the emergence of the Internet, "stuff related to IT" has been using more and more energy. It's like saying "5G won't use as much electricity as we are told! In fact 5G is more efficient than 4G". Yep, except that 5G enables us to use a lot mo…

Nothing forces the rebound effect to dominate. Computers grow cheaper, we rebound by buying ones with higher capacity, but the overall price still shrinks. I bet the computer you used to post today cost much less than Colossus. Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead. You can stream films via 5G that you might not have done via 4G, but you might've streamed via W…

>Nothing forces the rebound effect to dominate.

Human nature does. We're like a gas, and we fill to expand the space we're in. If technology uses less power, in general, we'll just use more of it until we hit whatever natural limits are present. (usually cost, or availability) I'm not sure I'm a proponent of usage taxes, but they definitely have the right idea; people will just keep doing more things until it becomes too expensive or they are otherwise restricted. The problem you run into is how the public reacts when "they" are trying to force a bunch of limitations on you that you didn't previously need to live with. It's politically impossible, even in a case where it's the right choice.

Re: AI won't use as much electricity as we are told (2024)

#57

Post by timschmidt »

bilekas wrote: post #32
Arnt wrote: post #29

Earlier quoted context omitted.

Nothing forces the rebound effect to dominate. Computers grow cheaper, we rebound by buying ones with higher capacity, but the overall price still shrinks. I bet the computer you used to post today cost much less than Colossus. Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead. You can stream films via 5G that you might not have done via 4G, but you might've streamed via W…

> Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead Am I missing something or has the need to vast GPU horsepower been solved ? Those requirements were not in DC's before and they're only going up. Whatever way you look at it, there's got to be an increase in power consumption somewhere no ?

GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt.

AI is still very near the beginning of the optimization process. We're still using (relatively) general purpose processors to run it. Dedicated accelerators are beginning to appear. Many software optimizations will be found. FPGAs and ASICs will be designed and fabbed. Process nodes will continue to shrink. Moore will continue to exponentially decrease costs over time as with all other workloads.

Re: AI won't use as much electricity as we are told (2024)

#58

Post by SketchySeaBeast »

dheera wrote: post #25

Even if AI doesn't use more electricity, electric cars and clean energy flight will need it.

That's my current, probably misguided hope. They couldn't justify getting the grid ready for electrical vehicles, but frame it as a way to make a bunch of money and everyone's going to jump on board.

Of course, the fact that xAI is throwing up gas turbines at their data centres seems to indicate that clean energy isn't a given.

Re: AI won't use as much electricity as we are told (2024)

#59

Post by bob1029 »

I'd be very interested in seeing some kind of aggregated daily demand curve for AI workloads.

It seems like a lot of the hyperbolic angles are looking at this as a constant draw of power over time. There is no reason for a GPU inference farm to be ramped up to 100% clock speed when all of its users are in bed. The 5700XT in my computer is probably pulling a mere 8~12W right now since it is just sitting on an idle desktop. A hyperscaler could easily power down entire racks based upon anticipated demand and turn that into 0W.

Re: AI won't use as much electricity as we are told (2024)

#60

Post by slfnflctd »

Tycho wrote: post #47

What’s the energy profile of running inference in a typical ChatGPT prompt compared to: - doing a google search and loading a linked webpage - taking a photo with your smartphone and uploading it to social media for sharing - playing Fortnite for 20 minutes - hosting a Zoom conference with 15 people - sending an email to a hundred colleagues I’d be curious. AI inference is massively centralised, so of course the data…

These are the kinds of questions we need pursued to develop better insight into the overall societal impact of current and near-future LLMs. Energy usage is a critical measure of any technology. The tradeoffs between alternate use cases should be modeled as accurately as possible, including all significant externalities.
Post reply on HN