Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral reports on the environmental impact of LLMs

mistral.ai

Post reply on HN
Print view View original

41–50 of 59 posts

Re: Mistral reports on the environmental impact of LLMs

#41

Post by chessgecko »

kingstnap wrote: post #30
greyadept wrote: post #2

I would really like it if an LLM tool would show me the power consumption and environmental impact of each request I’ve submitted.

You can assume the API price is roughly proportional to electricity usage. If you buy $10 in tokens, that probably folds into ~$3 to $5 dollars in electricity. Which would be around 30 to 90 kWhr in electricity. Depending on the source, it could be anywhere from ~500g/kWhr (for natural gas) and ~24g/kWhr for hydroelectric. It's a really wide spread, but I'd say for $10 in tokens, you'd probably be in the neighbourhoo…

if you bought a nvidia h100 at wholesale prices (around $25k) and ran it 24/7 at commercial electric rates (lets say $0.1 per kwh), then it would take you over 40 years to spend the purchase price of the gpu in electricity. Maybe bump it down to 20 for data center cooling.

I don't think the cost of the ai is close to converging to the price of power yet. Right now its mostly the price of hardware and data center space minus subsidies.

Re: Mistral reports on the environmental impact of LLMs

#42

Post by Dylan16807 »

jrflowers wrote: post #31
j-pb wrote: post #20

Earlier quoted context omitted.

You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My…

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

You're writing a whole lot of words that completely ignore they already gave a solution to things being cumulative.

Skip meat for one day, use AI for a year, come out ahead.

Re: Mistral reports on the environmental impact of LLMs

#43

Post by jrflowers »

j-pb wrote: post #35
jrflowers wrote: post #31

Earlier quoted context omitted.

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

Whatever you need to tell yourself to keep eating meat buddy.

> Whatever you need to tell yourself to keep eating meat buddy.

I’m not the one that brought up moralizing or food. I can’t really comment on your relationship with your diet but it kind of seems like you saw somebody mention power usage and unprompted shared “well I don’t eat meat or cheese or yogurt” so I guess keep that up while you use enough energy to power your home to write some code slower than you would without it?

https://www.infoworld.com/article/4020931/ai-coding-tools-ca...

I gave a little more detail to my point about climate impact here

https://news.ycombinator.com/item?id=44654552

Re: Mistral reports on the environmental impact of LLMs

#44

Post by jrflowers »

Dylan16807 wrote: post #42
jrflowers wrote: post #31

Earlier quoted context omitted.

>1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant You make a good point. A problem is only a real problem if you can’t find a bigger thing that makes it look small by comparison. For example, the worldwide concrete industry creates more co2 than beef does so there is no reason to stop eating beef if you en…

You're writing a whole lot of words that completely ignore they already gave a solution to things being cumulative. Skip meat for one day, use AI for a year, come out ahead.

They just suggested a different bike shed — one for the purpose of their argument won’t ever get fixed. J-pb’s point is that running a bunch of generators 24/7 in Memphis is fine because people eat meat. Inefficient LLMs in the real world are okay because people could theoretically become vegan but have not. It’s just a thought experiment

https://futurism.com/elon-musk-memphis-illegal-generators

Re: Mistral reports on the environmental impact of LLMs

#45

Post by Dylan16807 »

jrflowers wrote: post #44
Dylan16807 wrote: post #42

Earlier quoted context omitted.

You're writing a whole lot of words that completely ignore they already gave a solution to things being cumulative. Skip meat for one day, use AI for a year, come out ahead.

They just suggested a different bike shed — one for the purpose of their argument won’t ever get fixed. J-pb’s point is that running a bunch of generators 24/7 in Memphis is fine because people eat meat. Inefficient LLMs in the real world are okay because people could theoretically become vegan but have not . It’s just a thought experiment https://futurism.com/elon-musk-memphis-illegal-generators

If something costs too much, and you find a way to completely pay for it, that's not bikeshedding.

And it's not a thought experiment. It's a very real suggestion. If you're worried about the resource cost from your personal use, doing something to 100% offset it lets you stop worrying.

> become vegan

For one day per year. Replacing a day you would have otherwise eaten meat. That is an extremely attainable action for anyone that cares enough about LLM resource use enough to strongly consider avoiding them. It's not something that "will not change".

By the way, your goal of running efficiently on consumer hardware isn't as great as it sounds. One of the best ways to improve efficiency is batching multiple requests, and datacenter hardware generally uses more efficient nodes and runs at more efficient clock speeds. There's an efficiency sweet spot where models are moderately too big to run at home.

And it really undermines your argument when you throw in this stupid strawman about elon's toxic generators. You know j-pb was talking about typical datacenter resource use and not that. Get that insulting claim out of here.

Re: Mistral reports on the environmental impact of LLMs

#46

Post by jrflowers »

Dylan16807 wrote: post #45
jrflowers wrote: post #44

Earlier quoted context omitted.

They just suggested a different bike shed — one for the purpose of their argument won’t ever get fixed. J-pb’s point is that running a bunch of generators 24/7 in Memphis is fine because people eat meat. Inefficient LLMs in the real world are okay because people could theoretically become vegan but have not . It’s just a thought experiment https://futurism.com/elon-musk-memphis-illegal-generators

If something costs too much, and you find a way to completely pay for it, that's not bikeshedding. And it's not a thought experiment. It's a very real suggestion. If you're worried about the resource cost from your personal use, doing something to 100% offset it lets you stop worrying. > become vegan For one day per year. Replacing a day you would have otherwise eaten meat. That is an extremely attainable action for…

>It's a very real suggestion

It is only a “very real suggestion” if you believe that your argument might be effective.

Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact?

Because if not then you agree with me that in this case theoretical vegans are just being used to justify more real consumption, not less

>stupid strawman about elon's toxic generators

They exist in the real world, right now. It is a real phenomenon and no matter how many vegans I imagine it’s still there. I’m not really clear on why the real thing that’s really happening is a strawman unless you think that the existence of that system is so bad that it undermines your position. Even then it wouldn’t be a strawman though, just a thing that doesn’t support your position that using LLMs is categorically fine because you can picture a vegan in your head

Re: Mistral reports on the environmental impact of LLMs

#47

Post by Dylan16807 »

jrflowers wrote: post #46
Dylan16807 wrote: post #45

Earlier quoted context omitted.

If something costs too much, and you find a way to completely pay for it, that's not bikeshedding. And it's not a thought experiment. It's a very real suggestion. If you're worried about the resource cost from your personal use, doing something to 100% offset it lets you stop worrying. > become vegan For one day per year. Replacing a day you would have otherwise eaten meat. That is an extremely attainable action for…

>It's a very real suggestion It is only a “very real suggestion” if you believe that your argument might be effective. Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact? Because if not then you agree with me that in this case theoretical vegans are just being used to justify more real consumption, not less >stupid strawman about elon's toxic generators They exist in the real wor…

> Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact?

If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively).

I'd be tempted to write off both of those, but the whole point of your argument is to consider LLM resource use important, so I'm completely accepting that for the sake of the above argument.

There are no theoretical vegans involved.

And the suggestion doesn't even involve vegans, unless there's a massive contingent of americans that only eat meat one day per year that I wasn't aware of.

And to get at what I think is your core objection: The fact that people can do this isn't being used to let companies off the hook. If only 2% of LLM users set up a meat skipping day, then LLM companies are only 2% let off the hook.

But at the same time let's keep a proportional sense of how big the hook is.

> They exist in the real world, right now. It is a real phenomenon

The strawman is you accusing people of supporting those generators.

> your position that using LLMs is categorically fine

I didn't say that.

Re: Mistral reports on the environmental impact of LLMs

#48

Post by jrflowers »

Dylan16807 wrote: post #47
jrflowers wrote: post #46

Earlier quoted context omitted.

>It's a very real suggestion It is only a “very real suggestion” if you believe that your argument might be effective. Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact? Because if not then you agree with me that in this case theoretical vegans are just being used to justify more real consumption, not less >stupid strawman about elon's toxic generators They exist in the real wor…

> Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact? If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively). I'd be tempted to write off both of those, but the whole point of your argument is to consider LLM resource use important, so I'm completely acceptin…

>If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively).

Sorry, I should have clarified. In this case I meant “argument” as a thing that leads real people to either understand or agree with your position, not the construction of an idea in your mind.

With that in mind, do you think that “skip meat for a day use LLMs for a year” will convince enough real people, in real life, to not eat meat, that it offsets the emissions from LLM use?

Like imagine the future.

Since LLM use is a new category of energy use, you would have to convince people that haven’t already been convinced to skip meat by animal cruelty, health, philosophy, or existing climate concerns. People that were vegan before LLMs became popular obviously don’t count. The group of people that resisted decades of all that messaging will now make a meaningful adjustment to their consumption to cancel that out — and there will be enough of these new part time/full time vegans that it offsets the entire chat bot industry’s energy usage.

Do you imagine that being what happens?

If not it’s just somebody advocating for increased consumption in real life by invoking imaginary vegans.

As somebody that’s spent years as a vegan I am incredibly wary of “vegans can recruit” as a pitch. I’ve only ever heard that from people that have never tried to recruit in earnest or charlatans. Like I’ve mostly heard that from people that are not, never have been, and have no interest in being vegan.

Edit:

>The strawman is you accusing people of supporting those generators.

That’s not what a strawman is and it’s not an accusation, it’s an observation. If you say “I want subscription based online batched mega-high-compute language models” you are advocating for that industry, and those generators are part of it. Saying you feel that they’re somehow special and different because they’re icky does not make them any different from the thing that you say is necessarily the future. That you want!

Re: Mistral reports on the environmental impact of LLMs

#49

Post by Dylan16807 »

jrflowers wrote: post #48
Dylan16807 wrote: post #47

Earlier quoted context omitted.

> Do you believe that “skip meat for a day use LLMs for a year” will have a climate impact? If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively). I'd be tempted to write off both of those, but the whole point of your argument is to consider LLM resource use important, so I'm completely acceptin…

>If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively). Sorry, I should have clarified. In this case I meant “argument” as a thing that leads real people to either understand or agree with your position, not the construction of an idea in your mind. With that in mind, do you think that “skip mea…

I think anyone that does get convinced and skip meat should be able to use LLMs without shame or guilt, while we continue to pressure everyone else to save resources and we continue to pressure LLM companies to save resources.

LLM companies only get let off the hook if a very large fraction of their users do the meat skip thing, which is not very likely but could theoretically happen.

LLMs being a new category of energy use should get them some extra scrutiny, but only some. Maybe 3x scrutiny per wasted kilowatt hour compared to entrenched uses? If our real motivation is resource use, and not overreacting to change, LLMs should get some pressure but most of the pressure should go toward preexisting wasteful uses.

Nobody is advocating to ignore LLMs. But we shouldn't overstate them too much either.

And the giving up meat defense is not a defense for the companies, it's a defense for individual users that actually do it.

Re: Mistral reports on the environmental impact of LLMs

#50

Post by jrflowers »

Dylan16807 wrote: post #49
jrflowers wrote: post #48

Earlier quoted context omitted.

>If "use LLMs for a year" is enough to count as having a climate impact (negatively), then yes I believe "skip meat for a day use LLMs for a year" is enough to count (positively). Sorry, I should have clarified. In this case I meant “argument” as a thing that leads real people to either understand or agree with your position, not the construction of an idea in your mind. With that in mind, do you think that “skip mea…

I think anyone that does get convinced and skip meat should be able to use LLMs without shame or guilt, while we continue to pressure everyone else to save resources and we continue to pressure LLM companies to save resources. LLM companies only get let off the hook if a very large fraction of their users do the meat skip thing, which is not very likely but could theoretically happen. LLMs being a new category of ene…

It was a yes or no question

Like not an if or maybe thing, what do you see when you picture the future?

Do you think “Skip meat for a day use LLMs for a year” will produce enough new vegans to offset the energy usage and co2 produced by the LLM architecture of your choice?

Not asking if you want it to happen or if it’s something you can imagine could happen, I’m asking if you think it will

[_] yes

[_] no

Because if no, then the idea is just advocating for increased real consumption by invoking imaginary vegans!

Edit:

>LLM companies only get let off the hook if a very large fraction of their users do the meat skip thing, which is not very likely but could theoretically happen.

The person I was initially talking to took the position that LLM companies have negligible impact because people can be vegan. J-bp was saying that LLM companies shouldn’t be on anybody’s radars because uh, meat is 100,000 times worse.

The person you hopped in to defend was saying that LLM companies do not and should not have a “hook” because meat eaters exist

Post reply on HN