Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using ChatGPT is not bad for the environment

andymasley.substack.com

Post reply on HN
Print view View original

291–300 of 337 posts

Re: Using ChatGPT is not bad for the environment

#291

Post by rainonmoon »

minimaxir wrote: post #31
hexage1814 wrote: post #4

To me the "ChatGPT is destroying the environment "card always felt somewhat like bad faith arguing from the anti-AI crowd trying to find any excuse for being against AI. Like, the same people who complained about "using AI is destroying environment" seemed to have no issue with boarding a plane which would emit a bunch of CO2 so that they can have a vacation in Europe or the like. Selective environmentalism.

It's less bad faith, more a meme that has become so prevalent that it's impossible to dispell as it's something too nuanced for social media. I've seen more than a few social media posts asking "do they really cut down a rainforest every time someone generates an AI image?"

I mean we can go back and forth all day with credulous clowns on both sides. There are certainly plenty of misguided AI advocates who think what they're using is magic.

Re: Using ChatGPT is not bad for the environment

#292

Post by simonw »

oneplane wrote: post #243
originalvichy wrote: post #242

Where in the world are you getting the numbers for how much video streaming uses energy? I am quite sure that just as with LLMs, most of the energy goes into the initial encoding of the video, and nowadays any rational service encodes videos to several bitrates to avoid JIT transcoding. Networking can’t take that much energy, unless perhaps we are talking about purely wireless networking with cell towers?

LLM Inference is still quite power-hungry, Video decoding with hardware acceleration is much more efficient. But we can do some estimates, heck, we can even ask GPT for some numbers. Say you want to do 30 minutes of video (h265) or 30 minutes of LLM inferencing on a generic consumer device, ignoring the source of the model or source of encoded video, you get about 4x difference: Energy usage for 30 minutes of H.265 d…

"I would estimate that the local llama3 inferencing uses less power than when done in a datacenter, because there simply is less power available locally"

Is this taking into account the fact that datacenter resources are shared?

Llama 3 on my laptop may use less power, but it's serving just me.

Llama 3 in a datacenter or more expensive, more power-hungry hardware is potentially serving hundreds or thousands of users.

Re: Using ChatGPT is not bad for the environment

#293

Post by simonw »

alganet wrote: post #245

The part on training is misleading and full of shit. Training is not a "one-time cost". There is an implied never-ending need for training. LLMs are useless (for one of their main purposes) if the models get stale. I can use Musk's own argument on this one. Each model is a plane, fully built, that LLM researchers made into a disposable asset destined to be replaced by a newly built plane on the next training. Just in…

> LLMs are useless (for one of their main purposes) if the models get stale.

I don't think that's entirely accurate. A lot of people deliberately continue to chose to use the older GPT-4 despite it not being updated since June 2023.

GPT-4o has had releases in May, August and November of 2024 - so about one every 3-4 months.

Anthropic's Claude 3.5 Sonnet was released in June and had a single update in October.

Personally I'd rather have a model with excellent summarization / tool using abilities that can look up recent facts about the world.

Re: Using ChatGPT is not bad for the environment

#294

Post by simonw »

nharada wrote: post #247
titanomachy wrote: post #105

The major players in AI are collectively burning 1-2 gigawatts, day and night, on research and development of the next generation of LLMs. This is as much as my city of a million people. The impact is real, and focusing on inference cost per query kind of misses the point. Every person who uses these tools contributes to the demand and bears some of the responsibility. Similar to how I have responsibility for the car…

Agreed, I feel like the main response seems to be "Does Not!", but it's reasonable to accept that a thing you like has a cost. We all emit carbon every day to do things we don't 100% need, and we should just be willing to admit there's a cost and try and move towards paying them. Personally, I'm not tripping too hard about datacenter energy long term because it's very easy to make carbon free (unlike say ICE cars or…

"But it would be nice to see some efforts to incentivize green energy for those datacenters instead of just saying "whatever" and powering them with coal."

The major datacenter operators all have very aggressive targets for clean energy which they have been mostly sticking to.

Google: https://sustainability.google/progress/energy/

AWS: https://sustainability.aboutamazon.com/products-services/aws...

Azure: https://azure.microsoft.com/en-us/explore/global-infrastruct...

Re: Using ChatGPT is not bad for the environment

#295

Post by simonw »

baobun wrote: post #281
andymasley wrote: post #277

Earlier quoted context omitted.

The training cost adds a minuscule amount to the marginal cost of each search. The best sources I can find on training GPT-4 say it used 50 GWh of energy to train. GPT-4 is being used for a lot more than ChatGPT, but even if we assume that ChatGPT is all it's used for if we divide the total cost of energy by the total number of ChatGPT searches it increases the cost of each search by 10% (I go into this in the post)…

Sorry but this doesn't add up and it's not fixable because the argument is broken and I doubt the numbers are even in the right order of magnitude when considered in context. (That's before considering that it also had the effect of you writing and sharing this post - what's the second-order-effecf of people seeing it increasing their use as result of that and resharing the same idea with their peers etc? What's the…

Sounds to me like you're rationalizing your laziness in researching and sharing more accurate numbers because you want to justify "the negative effects that you truly deep down know are there".

You appear to be arguing based on vibes.

Re: Using ChatGPT is not bad for the environment

#296

Post by advael »

mritterhoff wrote: post #265
shafyy wrote: post #261

Earlier quoted context omitted.

Even if grid the was 100% renewable, this does not mean that there's no environmental cost to producing electricity. As a society, we need to decide what is important and try to minize energy consumption for things that are not important. And shoving LLMs into every nook and cranny of every application, so just tech giants who run the data centers can make more money and some middle managers get automatic summaries o…

What's the upside of moralizing energy consumption, especially once it's 100% renewable. Why not just let the market decide? If I'm paying for it, why does anyone else get a say in how I use it?

Isn't that kind of a non-sequitur? The claim made was that renewable energy would still be a finite resource to some degree. It's possible that the available energy surplus will be too big for any decisions about usage to matter, but that's a strong claim and you're doing nothing to make it here.

A lot of people believe in a higher power. If trusting in this supposed "market" brings you comfort and clarity in a complicated world, I do not begrudge you it. But invoking it doesn't address the claim it's answering

It's also clear that "the market" does not care enough about environmental impact to even do stuff like remove the current significant fossil fuel subsidies present in most government budgets, nor stop individuals or organizations from consuming or selling said fuels, natural gas, or plastic products at massive scales, so it's unclear why it would allocate energy in a way that didn't deprive crucial priorities.

Like the theodicy on the invisible hand's problem of environmental collapse ain't lookin' good is all I'm saying

Re: Using ChatGPT is not bad for the environment

#298

Post by saaaaaam »

Liquix wrote: post #17

~90% of the plastic debris in the ocean comes from ten rivers [0]. eight are in china/SEA. millions and billions of single-use items are sitting in warehouses and on store shelves wrapped in plastic. even before the plastic is discarded, the factories these items are produced in dump metric tons of waste into the oceans/soil with little repercussion. point is, none of our "personal lifestyle decisions" - not eating m…

Isn’t it also the case that a lot of those plastics come from those rivers because the first world ships is plastic water to China to be processed (“recycled”) and that this is what skews the numbers like this?

Re: Using ChatGPT is not bad for the environment

#299

Post by alganet »

simonw wrote: post #293
alganet wrote: post #245

The part on training is misleading and full of shit. Training is not a "one-time cost". There is an implied never-ending need for training. LLMs are useless (for one of their main purposes) if the models get stale. I can use Musk's own argument on this one. Each model is a plane, fully built, that LLM researchers made into a disposable asset destined to be replaced by a newly built plane on the next training. Just in…

> LLMs are useless (for one of their main purposes) if the models get stale. I don't think that's entirely accurate. A lot of people deliberately continue to chose to use the older GPT-4 despite it not being updated since June 2023. GPT-4o has had releases in May, August and November of 2024 - so about one every 3-4 months. Anthropic's Claude 3.5 Sonnet was released in June and had a single update in October. Persona…

LLMs that can replace search (their primary and self-declared goal) cannot survive without repeated training. As I said, this is one of its main purposes.

The other main purpose (military application, surveillance, autonomous psyops) is also highly dependent on continous training. Without it, properly educated healthy humans can overcome its reasoning power very quickly.

All other user profiles are just cannon fodder. Companies don't give a fuck about people running older models. They'll do whatever they can to make you use a more recent one.

That's why I'm being provocative with the "let's stop training new shit" argument. I'm aiming for the heel.

Post reply on HN