I love Nystrom's writing, and he's so good at it because he's written so much. A huge part of the value of things is how we grow in the making of them, and I worry that in a world where we accept generative slop, we'll never have the opportunity to woodshed enough to become excellent at a craft.
I'm a good engineer because I've written tons of code, I've taken no shortcuts, and I've focused on improving over my many iterations. This has enabled me to be an effective steward of generative coding (etc) models, but will younger engineers ever get the reps necessary to get where I am? Are there other ways to get this knowledge and taste? Does anyone know or care?
We're in the anthropocene now, and while probably everyone who knows what that is understands we have the largest effect on the Earth, it also means we now also have the largest effect on ourselves. We're so, so bad at taking this seriously. We can unleash technology that idiocracies western civilization inside of a generation, I know this because we keep lunging towards it with ever increasing success. But we can't just shamble around and let Darwin awards sort things out. We have nukes and virology labs, not to mention a climate change crisis to deal with. If the US political system falls apart because Americans under 65 spend between 2-3 hours on social media a day, that's a failed state with a lot of firepower to shoot around haphazardly.
And why do we keep building things that enfeeble us? Did we need easier access to delivery food and car rides, or did we need easier access to nutritious food and more walkable neighborhoods? Did we need social media with effectively no protections against propaganda/misinformation? We know that cognitive ability and executive function decline with LLM use. Can it really be that we think we're actually too smart and we need to turn it down a notch?
There are actual problems to solve, and important software to write. Neither algorithmic feeds nor advertising platforms fall under those categories. LLMs are supposed to solve the problem of "not enough software"--Nystrom points at this explicitly with the Washington Department of Ecology ad. But we never had a "not enough software problem", rather we had a "not enough beneficial software" problem, i.e. we'd be in a way better place if our best minds weren't working on getting more eyeballs on more ads or getting the AI to stop undressing kids.
Generative AI isn't empowering us. We don't have people building their own OSes, (real, working) browsers, word processors and spreadsheet programs, their own DAWs or guitar amp modelers, their own Illustrators or Figmas. Instead you have companies squeezing their workers and contractors, while their products enshittify. You can't even run these things without some megacorp's say so, and how are you gonna buy time on the H100 farm when AI took your job?
I'm too tired to write a conclusion. I'm pretty sure we're fucked. But hey look, the cars drive themselves.