Things we learned about LLMs in 2024
simonwillison.net
Things we learned about LLMs in 2024
1–10 of 615 posts
Re: Things we learned about LLMs in 2024
#2Don’t forget that 2024 was also a record year for new methane power plant projects. Some 200 new projects in the US alone and I’d wager most of them are funded directly by big tech for AI data centres.
https://www.bnnbloomberg.ca/investing/2024/09/16/ai-boom-is-...
This is definitely extending the runway of O&G at a crisis point in the climate disaster when we’re supposed to be reducing and shutting down these power plants.
Update: clarified the 200 number is in the US. There are far more world wide.
Re: Things we learned about LLMs in 2024
#3Interesting, the article is not quite what I expected.
Re: Things we learned about LLMs in 2024
#4Don’t forget that 2024 was also a record year for new methane power plant projects. Some 200 new projects in the US alone and I’d wager most of them are funded directly by big tech for AI data centres. https://www.bnnbloomberg.ca/investing/2024/09/16/ai-boom-is-... This is definitely extending the runway of O&G at a crisis point in the climate disaster when we’re supposed to be reducing and shutting down these power…
The only thing that will stop this is for battery storage to get cheap and available enough that it can cover for renewables. If we are still building gas turbines it means that hasn’t happened yet.
AI is a red herring. If it wasn’t that it would be EV power demand. If it wasn’t that it would be reshoring of manufacturing. If it wasn’t that it would be population growth from immigration. If it wasn’t that it would be replacing old coal power plants reaching EOL.
Replacing coal with gas is an improvement by the way. It’s around half the CO2 per kWh, sometimes less if you factor in that gas turbines are often more efficient than aging old coal plants.
Re: Things we learned about LLMs in 2024
#5My fav part of the writeup at the end:
"""
LLMs need better criticism # A lot of people absolutely hate this stuff. In some of the spaces I hang out (Mastodon, Bluesky, Lobste.rs, even Hacker News on occasion) even suggesting that “LLMs are useful” can be enough to kick off a huge fight.
I like people who are skeptical of this stuff. The hype has been deafening for more than two years now, and there are enormous quantities of snake oil and misinformation out there. A lot of very bad decisions are being made based on that hype. Being critical is a virtue.
If we want people with decision-making authority to make good decisions about how to apply these tools we first need to acknowledge that there ARE good applications, and then help explain how to put those into practice while avoiding the many unintiutive traps.
"""
LLMs are here to stay, and there is a need for more thoughtful critique rather than just "LLMs are all slop, I'll never use it" comments.
Re: Things we learned about LLMs in 2024
#6Don’t forget that 2024 was also a record year for new methane power plant projects. Some 200 new projects in the US alone and I’d wager most of them are funded directly by big tech for AI data centres. https://www.bnnbloomberg.ca/investing/2024/09/16/ai-boom-is-... This is definitely extending the runway of O&G at a crisis point in the climate disaster when we’re supposed to be reducing and shutting down these power…
It would be really cool if big tech could find a new hyperscaler model that didn't also require offsetting the goals of green energy projects worldwide. Between LLM and crypto you'd swear they're trying to find the most energy-wasteful tech possible.
Re: Things we learned about LLMs in 2024
#7> I’ve heard from sources I trust that both Google Gemini and Amazon Nova charge less than their energy costs for running inference...
Then, several headings later:
> I have it on good authority that neither Google Gemini nor Amazon Nova (two of the least expensive model providers) are running prompts at a loss.
So...which is it?
Re: Things we learned about LLMs in 2024
#8> I’ve heard from sources I trust that both Google Gemini and Amazon Nova charge less than their energy costs for running inference... Then, several headings later: > I have it on good authority that neither Google Gemini nor Amazon Nova (two of the least expensive model providers) are running prompts at a loss. So...which is it?
Oh whoops! That's an embarrassing mistake, and I didn't realize I had that point twice.
They're not running at a loss. I'll fix that.
Re: Things we learned about LLMs in 2024
#9I'd love to read a semi-technical book on everything that we've learned about what works and what does not on LLMs.
Re: Things we learned about LLMs in 2024
#10> Some of those GPT-4 models run on my laptop
That's an indication that most business-sized models won't need some giant data center. This is going to be a cheap technology most of the time. OpenAI is thus way overvalued.