Live data from Hacker News

Cognitive Debt: When Velocity Exceeds Comprehension

rockoder.com

191–200 of 234 posts

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#191
post #129
post #53

Earlier quoted context omitted.

I don't remember exactly what I wrote and how the logic works, but I generally remember the broad flow of how things tie together, which makes it easier to drop in on some aspect and understand where it is code-wise.

There's code structure but then there's also code philosophy. The worst code bases I have to deal with have either no philosophy or a dozen competing and incompatible philosophies. The best are (obviously) written in my battle tested and ultra refined philosophy developed over the last ~25 years. But I'm perfectly happy to be working in code bases written even with philosophies that I violently disagree with. Just as…

I think this is well put. Cohesive philosophy, even if flawed, is a lot easier to work with than a patchwork of out-of-context “best practices” strewn together by an LLM

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#192
post #136

"The system they built feels slightly foreign even as it functions correctly." This is exactly the same issue that engineers who become managers have. You are further away from the code; your understanding is less grounded, it feels disconnected. When software engineers become agent herders their day-to-day starts to resemble more that of a manager than that of an engineer.

exactly, as a manager and a sometimes a developer, "vibe-coding" has been looking more and more as my day job (in a good way, it's good to not have to do all the dirty work for your pet projects) and it's all about having the same discipline in term of: * thinking about the big picture * knowing how you can verify that the code match the big picture. In both case, somtimes you are happily surprised, sometimes you dis…

Engineering is not "dirty work."

Management is not "engineering."

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#193
post #14
post #9

The whole premise of the post, that coders remember what and why they wrote things from 6 months ago, is flawed. We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."

Harder here doesn’t mean slower. Reading and understanding your own code is way faster than writing and testing it, but it’s not easy. AI tools don’t prevent people from understanding the code they are producing as it wouldn’t actually take that much time, but there’s a natural tendency to avoid hard work. Of course AI code is generally terrible making the process even more painful, but you where just looking at the…

The reason it’s hard is exactly because you have to do it in shorter time and without a feedback cycle that has you learn bit by bit, like when you’d write the code yourself. It has some similarity with short-term cramming for an exam, where you will soon forget most of it afterwards, as opposed to when you built up the knowledge and problem-solving exercise over a longer period of time.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#194

Earlier quoted context omitted.

Writing is better than reading, but doing is better than writing.

How does this apply to coding when the act of writing IS doing? Or do you mean like coding "on your own" versus following a tutorial for example?

Means writing code (doing) vs writing documentation / plans / project architecture documents and so on.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#196
I am seeing similar dynamics at work, but also in my graduate studies.

I am currently doing the OMSCS at Georgia Tech and taking Machine Learning (7641) which has always had a reputation for being difficult. I don't mind a challenge, but I feel that the AI policy creates a sense of permanent and unpayable cognitive debt and learning deficits.

The class has traditionally taken a "data-first approach" to ML, where instead of focusing on the details of the different algorithms, students must apply them to datasets and analyze their performance and trade-offs. There are four colossal end-to-end ML projects which culminate in an 8-page IEEE-style paper each. (I actually prefer this general direction rather than an algo-heavy one - I find it more valuable to my work in business applications.)

For their AI policy, they've decided that all code can be generated by AI - the only rule is that the paper contents must be original analysis. To avoid taking any risks, I do not even use spell-checking AIs on the paper.

However, it seems to me that to compensate for the AI help, they've cranked up the amount of ground that needs to be covered in the projects. In the first project we were given two datasets, six algos to test, and a bunch of params and metrics to experiment with, producing a real combinatorial explosion of stuff to work on. This is on top of up to around 150+ pages of scientific reading on some weeks.

I am leaning very heavily on LLMs to generate massive chunks of the code, but I feel like I can't keep up at all.

I don't even feel my skills coming in where poor. I am a confident programmer, recently brushed up on math, and this is actually my second CS degree and my fourth course at Georgia Tech. I am rather familiar with the feeling of difficult courses or work problems pushing me to my intellectual limits where I stare into the abyss, but this feels radically different.

I am pushed to work at a higher (less detailed) level of abstraction, as many have foretold LLMs would do. I feel like I am learning about the data science meta-process but cannot keep up with details that are not even that fine. There is some complex math in there that could probably make my head spin but I cannot even get to that - I am cognitively stuck at higher abstractions like keeping up with so many families of algos, datasets, APIs, and thousands of AI generated codes.

In some sense this may be a shape of things to come at work too, but here's where that analogy breaks down: the performance of our work doesn't matter and we're not even graded on it. As long as we convincingly explain why things happen, we should be good, but even as I start to get the class and focus on that, I feel like I can barely keep up. If only they had made a bit of room with the AI productivity increase to focus a bit longer on that!

I thought I was losing it but this morning I found a Reddit thread with dozens of current students venting and found some solace in seeing that I'm not alone.

I also feel for the teaching staff, who I think are absolutely well-meaning, competent and attentive, but who just like the rest of us are trying to wing it in this brave new world.

AI is transformative for the good and the bad, and it's going to take us all many years to sort it out. We're not even started understanding social media and AI could be orders of magnitude more complex and also further complicating the former.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#197
post #48

Not to disagree with anything the article talks about but to add some perspective... The complaint about "code nobody understands" because of accumulating cognitive debt also happened with hand-written code. E.g. some stories: - from https://devblogs.microsoft.com/oldnewthing/20121218-00/?p=58... : >Two of us tried to debug the program to figure out what was going on, but given that this was code written several year…

"when I wrote the code, only me and God understood it. Now, only God understands it."

(attributed to Martin Fowler but I can't find any solid evidence)

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#198
post #48

Not to disagree with anything the article talks about but to add some perspective... The complaint about "code nobody understands" because of accumulating cognitive debt also happened with hand-written code. E.g. some stories: - from https://devblogs.microsoft.com/oldnewthing/20121218-00/?p=58... : >Two of us tried to debug the program to figure out what was going on, but given that this was code written several year…

Probably, we need to start saving prompts in Version Control. Prompts could be the context for both humans & machines.

I've been doing a version of this in a side project. Instead of saving the prompt directly, I have a road map. When implementing features, I tell it to brainstorm implementation for the road map. When fixing a bug, I tell it to brainstorm fixes from the roadmap. There's some back and forth, and then it writes a slice that is committed. Then, I look it over, verify scope, and it makes a plan (also committed). Then it generates work logs as it codes.

My prompts are literally "brainstorm next slice" or "brainstorm how to fix this bug" or "talk me through trades offs of approach A Vs B" so those prompts aren't meaningful in their own.

It's quite effective, but I'm a team of one.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#199

I’ve found a good counter to this is having agents visualize and explain the architecture of the system. Then I gain just enough context to figure out what I’m trying to accomplish. Also, as always, a highly modular codebase is very important. If I only have to reason about a single module then I don’t have to have full context on system. It seems we’re now in a world where engineers are responsible for creating a go…

> It seems we’re now in a world where engineers are responsible for creating a good environment where an agent is able to gain context on the architecture and validate its work via tests Part of me feels like we could have increased both velocity and comprehension a great amount twenty years ago already if we'd only had the same considerations for our fellow developers.

Fully agree. Perhaps I should have clarified that it’s primarily for agents now, not just engineers.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#200
post #189

Earlier quoted context omitted.

I definitely understand my own code better than what other people wrote, even from 10 years ago. I often see code and think "this makes sense to do it this way". Turns out I wrote it years ago.

That means your personal growth has plateaued. ;)

My coding style definitely has plateaued. It probably even has regressed to preferring simpler code.
Post reply on HN