this seems like one of those nonsense posts people will look at in a couple years and laugh at
Cognitive Debt: When Velocity Exceeds Comprehension
81–90 of 234 posts
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#82I think we might as well just go all in at this point: "LGTM, LLM". The industry always overshoots and then self-corrects later. Therefore, maybe the right thing to do is help it get to a more sane equilibrium is to forget about the code altogether and focus on other ways to constrain it / ensure correctness and/or determine better ways to know when comprehension is needed vs optional. What I don't like is the imposs…
The productivity boost entirely depends on the way the software was written.
Brownfield legacy projects with god classes and millions of lines of code which need to behave coherently across multiple channels- without actually having anything linking them from the written code? That shit is not even gonna get a 20% boost, you'll almost always be quicker on your own - what you do get is a fatigue bonus, by which I mean you'll invest yourself less for the same amount of output, while getting slightly slower because nobody I've ever interacted with is able to keep such code bases in their mind sufficiently to branch out to multiple agents.
On projects that have been architected to be owned by an LLM? Modular modilith with hints linking all channels together etc? Yeah, you're gonna get a massive productivity boost and you also will be using your brain a shitton actually reasoning things out how you'll get the LLM to be able to actually work on the project beyond silly weekends toy project scope (100k-MM LOC)
But let's be real here, most employees are working with codebases like the former.
And I'm still learning how to do the second. While I've significantly improved since I've started one year ago, I wouldn't consider myself a master at it yet. I continue to try things out and frequently try things that I ultimately decide to revert or (best case) discard before merge to main simply because I ... Notice significant impediments modifying/adding features with a given architecture.
Seriously, this is currently bleeding Edge. Things have not even begun to settle yet.
We're way too early for the industry to normalize around llms yet
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#83The whole premise of the post, that coders remember what and why they wrote things from 6 months ago, is flawed. We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."
Oh come on, that is complete nonsense. I can reunderstand complicated code I wrote a year ago far, far faster than complicated code someone else wrote. Especially if I also wrote tests, accompanying notes, and docs. If you can't understand your old code when you come back to it... including looking through your comments and docs and tests... I'm going to say you're doing it wrong . Maybe it takes a while, but it shou…
The trade-off is worth it in my opinion when you are in a time crunch to deliver a demo, or are asked to test out an idea for a new feature (also in a time crunch).
The hope being that if the feature were to be kept or the demo fleshed out, developers would need to shape and refactor the project as per newly discovered requirements, or start from scratch having hopefully learnt from the agentic rush.
To me, it always boils down to LLMs being probabilistic models which can do more of the same that has been done thousands of times, but also exhibit emergent reasoning-like properties that allow them to combine patterns sometimes. It's not actual reasoning, it's a facsimile of reasoning. The bigger the models, the better the RLHF and fine-tuning, the more useful they become but my intuition is that they'll always (LLMs) asymptotically try to approach actual reasoning without being able to get there.
So the notion of no-human-brain-in-the-loop programming is to me, a fool's errand. I do obviously hope I am right here, but we'll see. Ultimately you need accountability and for accountability you need human understanding. Trying to move fast without waiting for comprehension to catch up (which would most likely result in alternate, better approaches to solving the problem at hand) increases entropy and pushes problems further down the road.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#84The article very much resonates with my experience past several months.
The project I work on has been steadily growing for years, but the amount of engineers taking care of it stayed same or even declined a bit. Most of features are isolated and left untouched for months unless something comes up.
So far, I managed growing scope by relying on tests more and more. Then I switched to exclusively developing against a simulator. Checking changes with real system become rare and more involved - when you have to check, it's usually the gnarliest parts.
Last year's, I noticed I can no longer answer questions about several features because despite working on those for a couple of months and reviewing PRs, I barely hold the details in my head soon afterwards. And this all even before coding agents penetrated deep into our process.
With agents, I noticed exactly what article talks about. Reviewing PR feels even more implicit, I have to exert deliberate effort because tacit knowledge of context didn't form yet and you have to review more than before - the stuff goes into one ear and out of another. My team mates report similar experience.
Currently, we are trying various approaches to deal with that, it it's still too early to tell. We now commit agent plans alongside code to maybe not lose insights gained during development. Tasks with vague requirements we'd implicitly understand most of previously are now a bottleneck because when you type requirements to an agent for planning immediately surface various issues you'd think of during backlog grooming. Skill MDs are often tacit knowledge dumps we previously kept distributed in less formal ways. Agents are forcing us to up our process game and discipline, real people benefit from that too. As article mentioned, I am looking forward to tools picking some of that slack.
One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate. It's as if the concept was alien to them or they could comprehend that other people handle that at different capacity than them.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#85I have a view that we are shifting from the traditional form of Engineering into a more AI guided form, where may be we are not learning as much about the code but about how we can produce that code with correct instructions and high level design. It's like how we might not know how sewing is done but we know how to put instructions in a loom to produce it. I also agree it is still important to read that code and und…
Things always eventually fail. To be able to understand everything related is, and will be a massive advantage to every aspect of existence.
We shouldnt be giving it up just for some mild convenience (which seems so far overall really mild). The gain simply doesnt match the long term loss.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#86This seems very similar to the situation of a new employee dropped into a large codebase of varying quality. It seems like similar techniques will get you out of the mess? Also, you can ask the coding agent for help at understanding it, unlike the old days when whoever wrote it is long gone.
Only the coding agent will only give you plausible answers, not necessarily correct ones. So read the code. Oh, but you now can't because all you know is how to ask coding agents.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#87The solution is ironically, LLMs. You can construct a set of Claude skills to walk you through a code review, or understand code (anything, even course) fast.
This complexity to understanding compression will be a big market going forward.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#88The whole premise of the post, that coders remember what and why they wrote things from 6 months ago, is flawed. We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."
Understanding other people's code is harder than understanding your own code though.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#89Great article. I agree with the argument. But to offer a counter argument, would the same thing not have happened with the rise of high level languages? The machine code was abstracted away from engineers and they lost understanding of it, only knowing what the high level code is supposed to do. But that turned out fine. Would llms abstracting the code away so engineers only understand the functionality (specs, tests…
Yes my hot take is that the real risk isn't skill atrophy... it's failing to develop the new skill of using AI. It's all abstraction layers anyway and people always lament the next abstraction up.
0/1s → assembly → C → high-level languages → frameworks → AI → product
The engineer keeps moving up the abstraction chain with less and less understanding of the layers below. The better solution would be creating better verification, testing, and determinism at the AI layer. Surely we'll see the equivalent of high-level languages and frameworks for AI soon.
Re: Cognitive Debt: When Velocity Exceeds Comprehension
#90The solution is ironically, LLMs. You can construct a set of Claude skills to walk you through a code review, or understand code (anything, even course) fast. This complexity to understanding compression will be a big market going forward.
[flagged]