Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bottle of water per email: the hidden environmental costs of using AI chatbots

washingtonpost.com

Post reply on HN
Print view View original

41–50 of 69 posts

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#41

Post by gcr »

With reports like this discussing training cost, is the usage reported per-model? Or is it aggregated across the entire creative model development/exploration process?

At the risk of doing original research, one thing I don’t see a lot of discussion on is that AI companies don’t train one model at a time. Typical engineers will have maybe 5-10 mid-size models training at once. Large automated hyperparameter grid searches might need ensembles of hundreds or thousands of training runs to compare loss curves etc... Most of these will turn out to be duds of course. Only one model gets released, and that one’s energy efficiency is (presumably) what’s reported.

So we might have to multiply the training numbers by the number of employees doing active research, times the number of models they like to keep in flight at any given time.

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#43

Post by gcr »

yunwal wrote: post #37
gcr wrote: post #28

Earlier quoted context omitted.

So you can double your commute‘s environmental impact by using llama 1000x per day? That sounds pretty bad still, no?

A thousand times? I’d have a hard time typing out that many queries in 8 hours. Even 100 seems like a stretch for someone who uses it within something like cursor.

Coding models these days use an inference every time you stop typing. Let’s say it’s 0.1 inference oer keystroke. If you keep VSCode open all day, I could believe it’s a significant energy draw.

Google now uses several inferences per Google search.

The average user’s #inferences-per-day is going to skyrocket.

My point is that it’s understandable to consider AI a significant contributor to the average professional’s energy budget. It’s not an insult to point this out.

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#44

Post by yunwal »

poidos wrote: post #10
archerx wrote: post #3

I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.

Very high. But since they provide actual utility as opposed to the overwhelming majority of the current usecases for generative language models, it doesn’t seem relevant here.

Questionable in many cases. A bunch of stores have their A/C on full blast while keeping the door open in the summer. You can actually get fined for this in NYC now but in other US cities it’s still the norm.

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#46

Post by rainsford »

luma wrote: post #22

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping laptop while I spend the next 10 minutes penning an email? If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually inclu…

It would have been nice if the article did the comparison for us, but the the premise of the article seems correct. According to the article, ChatGPT uses 140 watt hours to compose a short email. That's considerably more than the battery capacity of many laptops, so even if you spent all day staring at your laptop trying to write the email, you're probably using less energy.

Now there's certainly a time advantage to using AI, but LLM based AI is pretty inefficient from an energy perspective.

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#47

Post by jedoch »

fhd2 wrote: post #26
luma wrote: post #22

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping laptop while I spend the next 10 minutes penning an email? If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually inclu…

That seems to get a bit philosophical. Are you not gonna use your laptop ~8 hours a day thanks to generative AI? If you still do, that part is constant. Are you gonna get less done in those 8 hours? Probably. Is it good for the environment to get more done in less time? Can't really answer that one generically. We humans do a lot of stuff simply because we can. If unemployment goes up 20% from generative AI (just a m…

You writing remids me, what the heck had us so damn distracted 20y ago, cos in front of google (if that may be a starting point ~20y ago) we had heard much of it, not? ...OT...but what was with the radiation levels caesium an uran that past day measured ...what is in norway...or...um...realy off-topic... (to downvote till deleted, please)^^

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#48

Post by spencerchubb »

I argue it's impossible to measure the total environmental impact of anything, and it's a waste of time to even try.

Let's break this down. There is water used at inference time and training time. The humans working on the project consume water. The building in which they did the project uses water. The whole supply chain for the computers? Good luck measuring the water usage in there

This may seem pedantic, but I promise there is a point to this.

Measuring environmental impact is like trying to understand a neural network.

If you want to discourage water usage, the only way is to tax the marginal water used at any step in the supply chain. You don't need to know the total water usage for the last step of the supply chain, in this case, an LLM

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#49

Post by archerx »

poidos wrote: post #10
archerx wrote: post #3

I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.

Very high. But since they provide actual utility as opposed to the overwhelming majority of the current usecases for generative language models, it doesn’t seem relevant here.

I lived in the tropics where it was hotter than the USA and we survived just fine without A/C.

A/C is luxury and your luxury is destroying the planet. I don’t want you to stop using A/C but I would like people to accept that it is a luxury that is not needed in most cases.

Re: A bottle of water per email: the hidden environmental costs of using AI chatbots

#50

Post by rainsford »

EraYaN wrote: post #25
luma wrote: post #22

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping laptop while I spend the next 10 minutes penning an email? If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually inclu…

The biggest problem there is that that laptop is pretty darn efficient. And multiple GPUs and other servers running 100% to run that query for you are not quite as efficient. It does so much more math to get the LLM to do anything useful, the scale is really staggering compared to registering your keystrokes and updating the screen, even if that all happens in a bloated Javascript code base. You can run that laptop,…

Per the article, the energy used by AI to write a short email (140 watt hours) is roughly equivalent to running a higher end desktop CPU flat out for an hour. That really puts the amount of computation going on behind the scenes of LLMs in perspective.
Post reply on HN