Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral reports on the environmental impact of LLMs

mistral.ai

Post reply on HN
Print view View original

31–40 of 59 posts

Re: Mistral reports on the environmental impact of LLMs

#31

Post by jrflowers »

j-pb wrote: post #20
jrflowers wrote: post #15

Earlier quoted context omitted.

This is spot on because there can’t be two issues that exist simultaneously. There can only be one thing that wastes enormous amounts of energy and that thing is beef

You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My…

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant

You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you enjoy it.

Now I know that some might say that “all of this is cumulative” or “the material problems that stem from entrenched industries is actually a reason not to invent completely novel wasteful things rather than a justification for them” but in reality only two things are true: only the biggest problem is real, and the only problem is definitely some other guy’s doing. If I waste x energy and my neighbor wastes y amount, a goal of reducing (x+y) is oppressive whereas a goal where I just need to try to keep x lower than y feels a lot nicer.

https://www.theguardian.com/cities/2019/feb/25/concrete-the-...

https://www.chathamhouse.org/sites/default/files/publication...

Re: Mistral reports on the environmental impact of LLMs

#32

Post by dijit »

kingstnap wrote: post #30
greyadept wrote: post #2

I would really like it if an LLM tool would show me the power consumption and environmental impact of each request I’ve submitted.

You can assume the API price is roughly proportional to electricity usage. If you buy $10 in tokens, that probably folds into ~$3 to $5 dollars in electricity. Which would be around 30 to 90 kWhr in electricity. Depending on the source, it could be anywhere from ~500g/kWhr (for natural gas) and ~24g/kWhr for hydroelectric. It's a really wide spread, but I'd say for $10 in tokens, you'd probably be in the neighbourhoo…

I don’t think that you can make this assumption.

People are selling AI at a loss right now.

Re: Mistral reports on the environmental impact of LLMs

#33

Post by AnimalMuppet »

jrflowers wrote: post #31
j-pb wrote: post #20

Earlier quoted context omitted.

You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My…

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

Human energy, attention, and effort are finite. Put the effort where it will have the biggest effect.

Re: Mistral reports on the environmental impact of LLMs

#34

Post by jrflowers »

AnimalMuppet wrote: post #33
jrflowers wrote: post #31

Earlier quoted context omitted.

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

Human energy, attention, and effort are finite. Put the effort where it will have the biggest effect.

I agree. Humans have been eating meat and doing construction for the entire history of civilization, they are not the sort of things that could be affected by posting online. LLMs on the other hand are new, largely in the hands of a small handful of companies, and a couple of those companies are bleeding cash in such a way that they might actually respond to consumer pressure. It is cynical to compare them to things that we know will not change as a justification for a blanket excuse for them.

Seeing as these models being wasteful is integral to the revenue of companies like OpenAI and Anthropic, the more people that tell them that the right business strategy is to start perpetually building data centers and power plants, the less incentive they have to build models that run efficiently on consumer hardware.

Re: Mistral reports on the environmental impact of LLMs

#35

Post by j-pb »

jrflowers wrote: post #31
j-pb wrote: post #20

Earlier quoted context omitted.

You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My…

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

Whatever you need to tell yourself to keep eating meat buddy.

Re: Mistral reports on the environmental impact of LLMs

#36

Post by kurthr »

evrimoztamur wrote: post #19

What is the levelised cost per token? As in how we calculate levelised cost of energy. If we take the total training footprint and divide that by the number of tokens the model is expected to produce over its lifetime, how does that compare to the marginal operational footprint? My napkin math says per token water and material footprints are up 6-600% and 4-400% higher respectively for tokens on the order of 40B to 4…

Within marginal error, dollars=destruction.

Even if the company is "green" they make money, they pay employees/stockholders, those people use the money to buy more things and go on vacations in airplanes. Worse, they invest the money to make more money and consume more goods.

Even your gains and vegetables are shipped in to feed you, if you walk to the grocery store. You pay rent/mortgage for a house built with concrete and steel. The highest priced items you pay for are also likely the most energy and environmentally costly. They create GDP.

It's a little weird with LLMs right now, because everything is subsidized by VC, Ads, BigCo investment so you can't see real costs. They're probably higher than the $30-200/mo you pay, but they're not 10x the price like your rent, car payment, food, vacation, investment/pension are.

Re: Mistral reports on the environmental impact of LLMs

#40

Post by tptacek »

bluefirebrand wrote: post #8
j-pb wrote: post #7

Earlier quoted context omitted.

people downvote your sarcasm, but if you do the calculations you're kinda right. 1Kg of Beef costs: - The energy equivalent of 60.000 ChatGPT queries. - The water equivalent of 50.000.000 ChatGPT queries. Applied to their metric Mistral Large 2 used: - The water equivalent of 18.8 Tons of Beef. - The CO2 equivalent of 204 Tons of Beef. France produces 3836 Tons of Beef per day, and one large LLM per 6 months. So yeah…

The difference is that food is important and live-giving and LLMs are a very fancy magic 8-ball

Where does a Youtube LetsPlay video fall into that calculation? My understanding is that a single watch of a video is orders of magnitude more than a day's active use of ChatGPT.
Post reply on HN