The Tragedy of the Cognitive Commons

I solved a Leetcode Medium on my first try in about 10 minutes the other day and it felt so amazing. Thinking through architecture, project composition, composability, etc when working with agents felt fine for a while but it made me feel disconnected from what I'm actually building.

Actually engaging my brain to solve the lower-level, on-the-ground code allows me to think of better ways to do things while I'm writing them. It's like writing anything. You start with something you want to convey, a thesis, and then it evolves and becomes better as you write it. An agent will just write it with no thought, as in it will reflect one of the LLM 'ghosts' as Andrej Karpathy puts it, doing something in the same way that someone in the training data has done it on a similar or different problem. This is why I get conniptions now when I am sent generated text or am expected to read it on a public forum. It's disrespectful of the time of every person expected to read it.

That said, I wouldn't like to go back to the before times without having the agentic option available. Ideally, businesses should not mandate how LLM's are to be used at their company, and just let the devs find their own flow. That is, if quality is even a factor that any company optimizes for anymore.

>I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.

It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm.

This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant.

What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics).

The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1.

Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot.

All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.

I think this is a really important idea. Good problem solvers, in general, are people who are prepared to go in and poke around, to see what happens when you disable this or add that.

The only way I've found to make juniors develop that instinct is to make them do it until it comes naturally--to continually demand they verify their assumptions, test out theories while debugging. If you don't practice being comfortable with the unfamiliar, you lose the knack.

It's fine, of course, to lose the knack to AI, as long as you'll never need to solve a problem the AI can't (or can't access). I don't think that's a good assumption for everyone to make.

The pseudo-academic style of the article is awful. All abstractions and buzzwords, no examples. Also, the tragedy of the commons concept is far older than 1968. The term "common" refers to common grazing land upon which anyone could graze animals. That's an English term. The American equivalent is "open range".

Getting past that, the author has a point. There's a loss of shared expertise when there aren't people around learning and doing something. In the US, we've seen this in manufacturing. The number of Americans who know how to set up a good production plant is much lower than it was in the 1980s. That's the consequence of the hollowing out of American manufacturing. The author talks about AI vs. white collar work, but fails to make the connection with outsourcing vs. blue collar work.

The loss of this expertise has recently been made very clear in the US as attempts are made to scale up weapons production for the US's various wars. Progress is very slow, as has been seen with both artillery ammo and air-defense missiles.

I don't know, I find AI still misses the mark so much for stuff that's not happening inside a well constrained framework with very specific guardrails and boundary conditions. Every time I have agents build something half complex that I would be easily able to pull off myself (albeit not at their pace) I always ask if everything is correct and working and I get a resounding "yes!", then I test it and it's essentially a steaming pile of garbage which I then tell the agent and after which it goes "you're absolutely right to push back, this isn't good at all!" as if it was clear from the beginning. I don't know if we'll achieve real intelligence soon with these things but what we have now isn't there yet. It's incredibly useful but I don't think you can replace a good programmer with it, you can make a good programmer into an excellent one though!

Same goes for writing, anyone who has written long complex texts with LLMs knows that a ton of editing is required to make it half decent.