Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Covering electricity price increases from our data centers

anthropic.com

Post reply on HN
Print view View original

31–40 of 110 posts

Re: Covering electricity price increases from our data centers

#31

Post by keeda »

wlesieutre wrote: post #22
keeda wrote: post #19

Earlier quoted context omitted.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

So with the AI is doing more of the work and you need less humans, what are you doing with the extra humans to eliminate their no-longer-productive resource consumption? Saying “we can do the same work with less resource use” doesn’t mean resource consumption is reduced. You’ve just gone from humans using resources to humans using the same resources and doing less work, plus AI using more resources.

The thing is, there are many interplaying dynamics here that are impossible to unravel. This is why I called it "napkin math", because figuring out the full ramifications of this change is a pretty large economic problem that nobody has figured out!

For instance, I think operating at this level of productivity is unsustainable (https://news.ycombinator.com/item?id=46938038). As discussed in detail by the recent "AI vampire" blog: https://news.ycombinator.com/item?id=46972179 -- most humans are not designed for that level of cognitive intensity.

But even then, the productivity per human will explode, and we will still have the problem of "too many humans." Cynically, if most knowledge workers get laid off, it's good from an environmental perspective because that means much less commuting and pollution! But then they're starving and we will have riots!

This is where I foresee the near-term problems with GenAI: social turmoil rather than resource consumption. I suspect it's not all bad news though. While it's impossible to put numbers on it, it helps to think about the first-order economic principles that are in play:

1. This is hand-wavy, but knowledge work boosts economic growth. If this is massively accelerated, we should be creating surplus value that compensates for a lot of costs.

2. However a huge chunk of knowledge work is busy work which will be automated away. People can try upskilling but the skill gap is already huge an growing quickly and they will lose jobs.

3. The economy is essentially people providing and paying for services and goods. If people lose jobs and cannot earn, they cannot drive the economy and it shrinks.

4. The elite, counter-intuitively enough, do NOT want that because they get richer by taking a massive cut of the economy! (Not to mention life in a doomsday bunker can get pretty dull if starving people start rioting -- https://news.ycombinator.com/item?id=46896066)

There are many more dynamics at play of course, but I think an equilibrium will be found purely because everyone is incentivized to find a solution (UBI?) that keeps both the elites and the plebes living long and prospering. I expect some turmoil, but luckily, the severe resource crunch of GPUs gives us time to figure things out.

Re: Covering electricity price increases from our data centers

#32

Post by keeda »

danielbln wrote: post #21
keeda wrote: post #19

Earlier quoted context omitted.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

Do keep in mind that 1 large prompt every 5 minutes is not how e.g. coding agents are used. There it's 1 large prompt every couple of seconds.

True, but I think in these scenarios they rely on prompt caching, which is much cheaper: https://ngrok.com/blog/prompt-caching/

I have no expertise here, but a couple years ago I had a prototype using locally deployed Llama 2 that cached the context (now deprecated https://github.com/ollama/ollama/issues/10576) from previous inference calls, and reused it for subsequent calls. The subsequent calls were much much faster. I suspect prompt caching works similarly, especially given changed code is very small compered to the rest of the codebase.

Re: Covering electricity price increases from our data centers

#33

Post by exabrial »

One of the potential upsides of AI in the USA is we'll bring down electrical prices compared to something like China. Power has to be abundantly plentiful and concentrated.

Maybe then, we could afford to smelt an ingot of aluminum in the USA.

Until then, I guess we're just sadly just burning coal to create cat memes. I hope Anthropic can lead the charge. Crypto was already a massive setback in terms of clean power, AI is already very dirty.

Re: Covering electricity price increases from our data centers

#34

Post by what »

keeda wrote: post #19
unltdpower wrote: post #3

"Committing to buying the glass to replace the window I broke in your shop to rob the place, you're welcome." > Training a single frontier AI model will soon require gigawatts of power, and the US AI sector will need at least 50 gigawatts of capacity over the next several years. These things are so hideously inefficient. All of you building these things for these people should be embarrassed and ashamed.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

How is a human consuming 27 gallons of water in an 8 hour work shift?

Re: Covering electricity price increases from our data centers

#35

Post by digitalPhonix »

esafak wrote: post #26
unltdpower wrote: post #3

"Committing to buying the glass to replace the window I broke in your shop to rob the place, you're welcome." > Training a single frontier AI model will soon require gigawatts of power, and the US AI sector will need at least 50 gigawatts of capacity over the next several years. These things are so hideously inefficient. All of you building these things for these people should be embarrassed and ashamed.

How are you measuring efficiency? They're better than most humans, which is what I would need more of as a substitute.

A human consumes about 100 watts when not doing any physical exertion (round number, rule of thumb). So unless you can show an LLM running on 100w compute with capabilities similar to a human, they’re less efficient.

Re: Covering electricity price increases from our data centers

#36

Post by DoctorOetker »

what wrote: post #34
keeda wrote: post #19

Earlier quoted context omitted.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

How is a human consuming 27 gallons of water in an 8 hour work shift?

Since their example scales the water consumption with their electricity consumption one may conclude it was the fresh water consumed (evaporated) during production of the electricity. Gaseous H2O is an even more potent GHG than CO2.

Re: Covering electricity price increases from our data centers

#37

Post by keeda »

what wrote: post #34
keeda wrote: post #19

Earlier quoted context omitted.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

How is a human consuming 27 gallons of water in an 8 hour work shift?

This includes things like drinking, sanitation, etc. Derived the number from here: https://www.epa.gov/watersense/statistics-and-facts

Mostly lines up with this reference too, which focuses only on water usage at work: https://quench.culligan.com/blog/average-water-usage-per-per...

Re: Covering electricity price increases from our data centers

#38

Post by pmontra »

digitalPhonix wrote: post #35
esafak wrote: post #26

Earlier quoted context omitted.

How are you measuring efficiency? They're better than most humans, which is what I would need more of as a substitute.

A human consumes about 100 watts when not doing any physical exertion (round number, rule of thumb). So unless you can show an LLM running on 100w compute with capabilities similar to a human, they’re less efficient.

100 W is only the start. Let's say that I consume 100 W all along the day. I use an LLM for coding assistance in the old way of asking questions and copy pasting code. It's much faster than me at writing that code. I don't think it ever works 1 hour for me per day. It's probably 10 cumulative minutes, probably much less. Round it up to 12 minutes to make it 1/5 of a hour or round it down to 6 minutes for a 1/10. So instead of 24 it's 0.1 hours, 240 times less. Those 100 W could be 24000 W and the total power per day would be the same. Is that LLM consuming 24 kW when working for me? No idea but I hope it's less than that.

Of course I could do all of my coding alone again, but I would be slower. It's like walking to the mall several times per week, several hours per time, instead of once or twice per week with a car, three cumulative hours. I trade a higher energy consumption for more time to do other things and the ability to live far away from shops.

Re: Covering electricity price increases from our data centers

#39

Post by dudisubekti »

unltdpower wrote: post #3

"Committing to buying the glass to replace the window I broke in your shop to rob the place, you're welcome." > Training a single frontier AI model will soon require gigawatts of power, and the US AI sector will need at least 50 gigawatts of capacity over the next several years. These things are so hideously inefficient. All of you building these things for these people should be embarrassed and ashamed.

Are you against inefficiency or just LLMs? If it's the former, I assure you LLMs are nowhere near the top of the list lol

You should start from beef industry.

Re: Covering electricity price increases from our data centers

#40

Post by pmontra »

wlesieutre wrote: post #22
keeda wrote: post #19

Earlier quoted context omitted.

> These things are so hideously inefficient. Quite the opposite, really. I did some napkin math for energy and water consumption, and compared to humans these things are very resource efficient. If LLMs improve productivity by even 5% (studies actually peg productivity gains across various professions at 15 - 30%, and these are from 2024!) the resource savings by accelerating all knowledge workers are significant. Si…

So with the AI is doing more of the work and you need less humans, what are you doing with the extra humans to eliminate their no-longer-productive resource consumption? Saying “we can do the same work with less resource use” doesn’t mean resource consumption is reduced. You’ve just gone from humans using resources to humans using the same resources and doing less work, plus AI using more resources.

Resource consumption often goes up. It's a time vs energy tradeoff and it's not free.

Your question is a variant of what do we do with all those humans now that they don't have to walk miles to the well every day because we invented aqueducts? The point is that they didn't want to walk to the well but they had to (and in some places they still have to) and very few people want to work, even now and even us, but they have to.

We will see what happens this time when we won't have to walk to that well.

Post reply on HN