Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding is the new bottleneck

geoffreylitt.com

Post reply on HN
Print view View original

231–240 of 259 posts

Re: Understanding is the new bottleneck

#231

Post by tpicks »

iamwil wrote: post #116
tpicks wrote: post #98

Earlier quoted context omitted.

same, I've been liking https://openspec.dev/ and find the more time I put iterating/scrutinizing the spec artifacts (proposal, requirements, design, tasks) before I let the agent implement the better understanding I have and the better results I get. Also like that it is agent agnostic so I can take it with me as I try different agents/models.

Has there been a time when your spec was accurate, the agent implemented it, everything passed — but you still felt you didn't really understand what was happening? If so, what did you do about it? Did it degrade your ability to make subsequent specs?

sure, if I get lazy and don't read every line of the spec or don't thoroughly review the generated code, then I have felt a gap in understanding, but thats more on me than the tools. Another case is when I'm just completely unfamiliar with the framework the code is written in, I feel this happen whenever I use AI to write any non-trivial frontend as I rarely did frontend pre-AI.

Re: Understanding is the new bottleneck

#232

Post by jagenabler2 »

TacticalCoder wrote: post #123
odshoifsdhfs wrote: post #71

Earlier quoted context omitted.

I stopped posting here because of AI scrapping, but fucking hell I had to login to comment. I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones). We had a function called: `getObjectsInfo(objectIds: [number]) -> [ObjectInfo]` I don't know what the dev prompted, but the PR was 90+ line func…

Exactly. Precisely. Definitely. I see what you describe all the time , because I do review the code the models do produce. It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense. And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wa…

The justification I’ve been getting from others is that future models will resolve all the tech debt, so paying a cost to iterate fast now is worth it.

We may be way past the point.

Re: Understanding is the new bottleneck

#233

Post by champagnepapi »

equinumerous wrote: post #149
nphardon wrote: post #130

In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the proble…

I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing soft…

Agreed!

Re: Understanding is the new bottleneck

#234

Post by lanstin »

Forgeties79 wrote: post #199
jfyi wrote: post #188

Earlier quoted context omitted.

I disagree, both of those are skills. Neither are skills that a large portion of users of those services pursue to any meaningful extent, I'll grant you that. They also certainly are not synonymous with the term "soft skills" as I know it. So I think I am on your side of the fence on that part any way.

It is impossible to duplicate results with an LLM. This strikes me as a serious barrier to calling it a proper skill. If you can’t even somewhat replicate the results you can’t really improve the input consistently. You can’t become “skilled” at it if you can’t even reproduce what you did. If you enter the same prompt 3 times the results are of pretty significantly different quality. ChatGPT literally has you A/B tes…

Me typing into emacs is also not replicable, until I git push. Me with emacs and a good coding LLM can generate systems that I see as good, in my extensive professional judgement. For coding, for production grade code, the LLM will mostly be a tool in the hands of the professional. There is a new category of disposable code, which I think will be useful for many adhoc investigations; in neither case is replicability a serious requirement.

I do a lot of guessing and validating in learning maths. It is pretty efficient way to build that conceptual understanding. I even try to predict the next big theorem as I am listening to a lecture or reading a text book. It's engaging.

And honestly, I am enjoying learning this new way to make code I am pleased with. Using the LLM effectively and for quality deliverables is different from typing in many surface ways, but modularity of thought, iterative design and implementation, simplicity and generality, documentation, all still pay off.

Re: Understanding is the new bottleneck

#235

Post by Kinrany »

neprotivo wrote: post #194

Improving code understanding is the main focus of my work and thinking right now. If we want to make advances I believe that we should rely more heavily on one key quality of the program code: It is meant to be executed. Here are some ideas: 1. Time travel debugging. Reading a PR just like a wall of text is difficult, but what if you could step through the PR and see the state at a given line for some test executions…

This seems backwards. We've known for a long time that successful execution is not enough for understanding.

Re: Understanding is the new bottleneck

#236

Post by orsenthil »

madrox wrote: post #82

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program ma…

> I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management.

This looks like we choose to hear what we want to hear.

The understanding argument is about making the point that "The point was always to augment" the human understanding when we worked with the machines.

Re: Understanding is the new bottleneck

#237

Post by euthymiclabs »

hackmack10 wrote: post #221
euthymiclabs wrote: post #2

"I read the code." -Mitchell Hashimoto Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.

Yeah, but in the corporate world now, you're being judged by how fast you get shit checked in. So, if you need to understand every line these days... you're going to end up in the bottom 25% of the company pretty quickly. Not saying you're wrong, just saying how fucked things are becoming.

Perverse incentives have always been the purview of bad management. Measuring things that don't cause success is easier than figuring out what actually matters. (It's also why I don't practice corporate medicine anymore.)

Re: Understanding is the new bottleneck

#238

Post by imtringued »

jagenabler2 wrote: post #232
TacticalCoder wrote: post #123

Earlier quoted context omitted.

Exactly. Precisely. Definitely. I see what you describe all the time , because I do review the code the models do produce. It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense. And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wa…

The justification I’ve been getting from others is that future models will resolve all the tech debt, so paying a cost to iterate fast now is worth it. We may be way past the point.

Do they actually iterate on their code? Or do they claim they are iterating?

Closing a ticket with more code doesn't count as iterating.

Re: Understanding is the new bottleneck

#239

Post by jagenabler2 »

imtringued wrote: post #238
jagenabler2 wrote: post #232

Earlier quoted context omitted.

The justification I’ve been getting from others is that future models will resolve all the tech debt, so paying a cost to iterate fast now is worth it. We may be way past the point.

Do they actually iterate on their code? Or do they claim they are iterating? Closing a ticket with more code doesn't count as iterating.

lol they're definitely not iterating on code. Instead they iterate on "ideas", with the expectation that code no longer matters because complexity will be self-resolving with agentic loops.

Unfortunately these tools, and the VCs/companies pushing to adopt them, has totally empowered this type of behaviour.

Re: Understanding is the new bottleneck

#240

Post by simianwords »

modo_mario wrote: post #230
simianwords wrote: post #166

Earlier quoted context omitted.

And being able to trust the agent when you can makes a good engineer

I think a good engineer still reads the majority of what llm's put out for him and understands it so trust doesn't really come into the picture.

no a good engineer doesn't have to read majority what LLM put out - its not the most efficient.
Post reply on HN