Live data from Hacker News

Cognitive Debt: When Velocity Exceeds Comprehension

rockoder.com

41–50 of 234 posts

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#41
post #33

Great article. I agree with the argument. But to offer a counter argument, would the same thing not have happened with the rise of high level languages? The machine code was abstracted away from engineers and they lost understanding of it, only knowing what the high level code is supposed to do. But that turned out fine. Would llms abstracting the code away so engineers only understand the functionality (specs, tests…

I think it won't be too different once we see a few upgrades that are going to be required for reliability (and scaling up the AI assisted engineering process):

  - deterministic agents, where the model guarantees the same output with a seed
  - much faster coding agents, which will allow us to "compile" or "execute" natural language without noticing the llm
  - maybe just running the whole thing locally so privacy and reliability are not an issue
We're not there yet, but once we have that then I agree there won't be too much of a difference between using a high level language and plain text.

There's going to be a massive shift in programming education though, because knowing an actual programming language won't matter any more than knowing assembly does today.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#42

This thread is closely related: https://news.ycombinator.com/item?id=47194847 "The right amount of AI is not zero. And it’s not maximum."

Author from the other thread here. I'm surprised to see so many similarities, but in good faith I'll assume that it's just a coincidence because many devs start to notice the upcoming problems.

I appreciate your good faith but I tried to copy-paste the first ~7k character of this (not yours!) article in an AI detector (gptzero) and it's "highly confident that this text was AI generated" with a probability of 100%

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#43
post #33

Great article. I agree with the argument. But to offer a counter argument, would the same thing not have happened with the rise of high level languages? The machine code was abstracted away from engineers and they lost understanding of it, only knowing what the high level code is supposed to do. But that turned out fine. Would llms abstracting the code away so engineers only understand the functionality (specs, tests…

I also was having a similar thought, and think you wrote the answer I could not put my finger on. Compilers are deterministic, AI is a stochastic process, it doesn't always converge exactly to the same answer. Here's the main difference

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#45
post #9

The whole premise of the post, that coders remember what and why they wrote things from 6 months ago, is flawed. We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."

Oh come on, that is complete nonsense. I can reunderstand complicated code I wrote a year ago far, far faster than complicated code someone else wrote. Especially if I also wrote tests, accompanying notes, and docs. If you can't understand your old code when you come back to it... including looking through your comments and docs and tests... I'm going to say you're doing it wrong . Maybe it takes a while, but it shou…

[deleted]

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#46
It reminds me of Clay Christensen’s book How to Measure Your Life. In one of his talks, he talked about how companies get killed because they optimized for the wrong/short-term metrics. What we are seeing with AI could be a supercharged flavor of Innovator’s Dilemma, where organizations optimize a pre-existing set of success metrics while missing the bigger picture because some previous assumptions no longer hold.

I really like the article. It’s not trying to sell fear (which does sell); it doesn’t paint the leaderships as clueless. Nobody knows what is going to happen in the future. The article might be wrong on a few things. But it doesn’t matter. It points out a few assumptions that people might be missing and that is great.

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#48
Not to disagree with anything the article talks about but to add some perspective...

The complaint about "code nobody understands" because of accumulating cognitive debt also happened with hand-written code. E.g. some stories:

- from https://devblogs.microsoft.com/oldnewthing/20121218-00/?p=58... : >Two of us tried to debug the program to figure out what was going on, but given that this was code written several years earlier by an outside company, and that nobody at Microsoft ever understood how the code worked (much less still understood it), and that most of the code was completely uncommented, we simply couldn’t figure out why the collision detector was not working. Heck, we couldn’t even find the collision detector! We had several million lines of code still to port, so we couldn’t afford to spend days studying the code trying to figure out what obscure floating point rounding error was causing collision detection to fail. We just made the executive decision right there to drop Pinball from the product.

- and another about the Oracle RDBMS codebase from https://news.ycombinator.com/item?id=18442941

(That hn thread is big and there are more top-level comments that talk about other ball-of-spaghetti projects besides Oracle.)

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#49
post #28

> The engineer who pauses to deeply understand what they built falls behind in velocity metrics. This is the most insidious part. It's not even that bad code gets deployed. That can be fixed and hopefully (by definition) the market weeds that out. The problem is that the market doesn't seem to operate like that, and instead the engineer who cares loses their job because they're not hitting the metrics.

Of course, there are counter examples but there's a disconnect between the production of something and the selling of it with almost opposing goals. Given unlimited money and time, many engineers, arts, etc will write and rewrite something to perfection. Constraints are needed because the world doesn't operate in a vacuum and unless we all live in a utopia, we have to compete for customers and resources.

Constraints often result in better results. Think of Duke Nukem Forever and how long it took them to release a nothingburger.

I just watched a show called the Knight of the Seven Kingdoms and the showerunners were given a limited budget compared to their cousin shows and it resulted in a better product.

Sometimes those metrics keep things on the rails

Re: Cognitive Debt: When Velocity Exceeds Comprehension

#50
This happened to me yesterday. I give a junior engineer a project. He turns it around really quickly with Cursor. I review the code, get him to fix some things (again turned around really quickly with Cursor) and he merges it. I then try a couple test cases and the system does the wrong thing on the second one I try. I ask him to fix it. He puts into cursor a prompt like "fix this for xyz case" and submits a PR. But when I look at the PR, it's clearly wrong. The model completely misunderstood the code. So I leave a detailed comment explaining exactly what the code does.

He's moving so fast that he's not bothering to learn how the system actually works. He just implicitly trusts what the model tells him. I'm trying to get him to do end-to-end manual testing using the system itself (log into the web app in a local or staging environment and go through the actions that the user would go through), he just has the AI generate tests and trusts the output. So he completely misses things that would be clear if you learned the system at a deep level and could see how the individual project you're working on fit in with the larger system

I see this with all the junior engineers on my team. They've never learned how to use a debugger and don't care to learn. They just ask the model. Sometimes they think critically about the system and the best way to do something, but not always. They often aren't looking that critically at the model's output.

Post reply on HN