A bottle of water per email: the hidden environmental costs of using AI chatbots
1–10 of 69 posts
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#2Is this true in a marginal cost sense? I was under the impression most of the environmental impact occurred during the training stage, and that it was significantly less costly post training?
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#3I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#4Is this true in a marginal cost sense? I was under the impression most of the environmental impact occurred during the training stage, and that it was significantly less costly post training?
Llama 403b takes OOM a kilowatt minute to respond on our local gpu server, or about 10 grams of C02 per email. Last I checked, add another 20 grams of amortized manufacturing emissions. A typical commute is OOM 5-10 kg of CO2.
this article is alarmist bullshit. (for entirely unrelated reasons openai delenda est)
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#5Is this true in a marginal cost sense? I was under the impression most of the environmental impact occurred during the training stage, and that it was significantly less costly post training?
You could argue that this is no longer the case once the model is done; the cost per request will go down over time, as the set amount of power and coolant pumped through data centres gets divided over more people.
However, AI companies can't afford to stand still. They have to keep training or they risk being made irrelevant by whatever AI company comes next.
Furthermore, a non-significant amount of energy and cooling is being used for generating responses as well. It's plainly obvious when you run even the very modest AI models at home how much power these things take.
The paper[1] mentions the statistics used to calculate these numbers. It has a separate column for inference, with numbers ranging from 10mL to 50mL of water per inference depending on the data centre sampled.
The numbers seem bad, but the authors also call out that more transparency is needed. With all the bad rep out there from independent estimations and no AI companies giving detailed environmental impact data, I have to assume the real cost is worse than estimated, or companies would've tried to greenwash themselves already.
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#6Does anyone else have a hard time accepting these calculations? I don’t doubt the serious environmental costs of AI but some of the claims in this infographic seem far-fetched. Inference costs should be much lower than training costs. And, if a 100-word email with GPT-4 requires 0.14 kWh of energy, power AI users and developers must be consuming 100x as much. Also, what about running models like Llama-3 locally? Would love to see someone with more expertise either debunk or confirm the troubling claims in this article. It feels like someone accidentally shifted a decimal point over a few places to the right.
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#7I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.
[deleted]
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#8Does anyone else have a hard time accepting these calculations? I don’t doubt the serious environmental costs of AI but some of the claims in this infographic seem far-fetched. Inference costs should be much lower than training costs. And, if a 100-word email with GPT-4 requires 0.14 kWh of energy, power AI users and developers must be consuming 100x as much. Also, what about running models like Llama-3 locally? Woul…
Even if the costs were lower, the trend is towards more inference compute time (o1), so these costs might be valid for the future.
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#9Does anyone else have a hard time accepting these calculations? I don’t doubt the serious environmental costs of AI but some of the claims in this infographic seem far-fetched. Inference costs should be much lower than training costs. And, if a 100-word email with GPT-4 requires 0.14 kWh of energy, power AI users and developers must be consuming 100x as much. Also, what about running models like Llama-3 locally? Woul…
If I run some simple inference locally on a 4090 (450 TDW card) it takes order of seconds and that sucker's going full blast, you're looking at order of 1 kJ, which is significantly higher than what is quoted in the article.
Article numbers line up better with CPU inference for ~1s.
Re: A bottle of water per email: the hidden environmental costs of using AI chatbots
#10I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.
Very high. But since they provide actual utility as opposed to the overwhelming majority of the current usecases for generative language models, it doesn’t seem relevant here.