Using ChatGPT is not bad for the environment

And to be honest what we need to do is replace them with nuclear power stations to manages the base load of nations power requirements. Either that or much better power storage is required

But we can do some estimates, heck, we can even ask GPT for some numbers.

Say you want to do 30 minutes of video (h265) or 30 minutes of LLM inferencing on a generic consumer device, ignoring the source of the model or source of encoded video, you get about 4x difference:

  Energy usage for 30 minutes of H.265 decoding: ~15–20 Wh.
  Energy usage for 30 minutes of Llama3 inference: ~40–60 Wh.

While not the most scientific comparison, it's perhaps good to know that video decoding is practically always local, and for streaming services it will use whatever is available and might even switch codecs (i.e. AV1, H.265, H.264 depending on what is available, and what licenses are used). And if you have older hardware, some codecs won't even exist in hardware, to the point where you start doing software decoding (very inefficient).

AI inferencing is mostly remote (at least the heavy loads) in a datacenter because local availability of hardware is pretty hit and miss, models are pretty big and spinning one up every time you just wanted to ask something is not very user friendly. Because in a datacenter you tend to pay for amperage per rack, you spec your AI inferencing hardware to eat that power since you're not saving any money or hardware life when you don't use it. That means that efficiency is important (more use out of a rack) but scaling/idling isn't really that big of a deal (but it has slowly dawned on people that burning power 'because you can' is not really a great model). That AI inferencing in a datacenter is more power-hungry as a result, because they can, because it is faster, and that's what attracts users.

I would estimate that the local llama3 inferencing uses less power than when done in a datacenter, because there simply is less power available locally (try finding an end-user device that is used mass-market with enough power available, you won't; only small markets like gaming PCs and workstations will do).

Training is not a "one-time cost". There is an implied never-ending need for training. LLMs are useless (for one of their main purposes) if the models get stale.

I can use Musk's own argument on this one. Each model is a plane, fully built, that LLM researchers made into a disposable asset destined to be replaced by a newly built plane on the next training. Just incredibly stupid and inneficient.

I know what you're thinking right now: fine-tuning, etc. That is the "reusable" analogy to that, is it not? But fine-tuning is far, far from reusability (the major players don't even care about it that much). It's not even on the "hopper" stage.

_Stop training new shit, and the argument becomes valid. How about that?_

---

I am sure the more radical environmentalists know that LLMs can be eco-friendly. The point is: they don't believe it will go that way, so they fight it. I can't blame them, this has happened before.

_This monster was made by environment promises that were not met_. If they're not met again, the monster will grow and there's nothing anyone can do about it. I've been more moderate than this article in several occations and still got attacked by it. If not LLMs, it will target something else. Again, can't blame them.

Personally, I'm not tripping too hard about datacenter energy long term because it's very easy to make carbon free (unlike say ICE cars or aircraft). But it would be nice to see some efforts to incentivize green energy for those datacenters instead of just saying "whatever" and powering them with coal.

A grid isn't a magic battery that is always there, it is constantly fluctuating, regardless of the intent of producers and consumers. You need to be able to have enough elasticity to deal with that fact. Changing that is hard (and expensive), but it is the only way (such as the technical reality).

The solution is not to create say, 1000 extra coal-fired generating facilities since you can't really turn them on or off at will. Same goes for gas, nuclear etc. You'd need a few of them for your baseline load (combined with other sources like solar, wind, hydro, whatever) and then make sure you have your non-renewable sources have margin and redundancy and use storage for the rest. This was always the case, and it will always be the case.

But now with information technology, the degree to which you can permanently raise demand on the grid to an extreme degree is where the problem becomes much more apparent. And because it's not manufacturing (which is an extreme consumer of energy) you don't really get the "run on lower output" option. You can't have an LLM do "just a little bit of inferencing". Just like you can't have your Netflix send only half a movie to "save power".

In the past we had the luxury of nighttime lower demand which means industry could up their usage, but datacenters don't sleep at night. And they also can't wait for batch processing during the day.