Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

-​-dangerously-skip-reading-code

olano.dev

Post reply on HN
Print view View original

171–180 of 219 posts

Re: -​-dangerously-skip-reading-code

#171

Post by skydhash »

theshrike79 wrote: post #159
runarberg wrote: post #135

Earlier quoted context omitted.

Simon Willison’s analogy does not apply unless that other team was immediately fired after they delivered the image resize service, or (more commonly) was done by a one off contractor. The difference is the trust model. We trust that our company has hired a competent team which maintains knowledge of the image resizing service, that they respond to bug reports and feature requests and that they know how to fix and im…

Why does the team need to be "fired"? The single person who did the service might just quit and go to another job. They might be external consultants that rotate away when the contract ends. It might be a SaaS service where you don't control the code at all - nor the composition of their team. We have trusted services, contractors and teams within our companies before. Now suddenly _everyone_ has ALWAYS read and meti…

As your parent comment says. It’s about trust. People don’t hire contractors with low reputations. Same with SaaS services. That’s why you see so much stuff about branding and customer testimonials. It can be gamed, but usually works well enough.

LLM have no reputation to lose. Their work may or may not be aligned with your goals and they can’t care if they messed up.

Re: -​-dangerously-skip-reading-code

#172

Post by skydhash »

zoogeny wrote: post #83
throawayonthe wrote: post #82

Earlier quoted context omitted.

can you explain how? with a compiler you can rely on the adage "it's never a compiler bug" (until it is! and then you can fix it) how can a local LLM with an open source agent harness provide the same trustworthiness?

> ... then you can fix it I recall working on a project that used (MSVC) VC++ and a coworker found a bug in the compiler. We reported the issue to Microsoft and they eventually patched it. You may find yourself arguing explicitly for open source dev tools if you continue down this line. There are many commercial cases where "you can fix it" does not apply to the dev toolchain and you will find yourself reliant on a p…

> There are many commercial cases where "you can fix it" does not apply to the dev toolchain and you will find yourself reliant on a provider.

That’s only on the hobbyist level. On the enterprise level, there are lots of contracts involved that requires speedy bugs correction.

Re: -​-dangerously-skip-reading-code

#173

Post by skydhash »

dpark wrote: post #94
tyleo wrote: post #80

Earlier quoted context omitted.

If leaking credentials is observable behavior. I don't get your point. Memory leaks, deleting the hard drive, spending money would all be observable behavior. By your reasoning that the "observable behavior needs to be specified rigorously" it seems like you'd have to list these all out. We do, after all, already have cases of AI deleting data. That sounds harder and more error prone than what we're doing now by rigo…

My ultimate point is that source code specifies everything the software can or will do, but most of that is not part of the required specification. Most of it is implementation details you don’t care about. The entire reason we have functions and components and modules etc is to isolate engineers from the things we do not need to care about. I should not need to care about the implementation details of most software,…

> I should not need to care about the implementation details of most software, only if it meets my retirements.

The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details that have not made it into the specifications. One one side you have the platform and on the other side you have the interface contract (requirements). Saying what’s in the middle doesn’t matter is strange. Because both the platform and the interface are dynamic and can shift drastically from their 1.0 version.

Re: -​-dangerously-skip-reading-code

#174

Post by StilesCrisis »

kenhwang wrote: post #132
jimmaswell wrote: post #123

Earlier quoted context omitted.

> every time I use AI for coding, to some capacity I'm sacrificing system understanding and stability in favor of programming speed. Sure, but couldn't you say the same for letting other people contribute code too? In either case, you make the choice of how deeply you want to review it. You can ask the AI or the human to explain things that aren't clear. For me it's case by case in either scenario. Sometimes it's not…

I always wondered why people don't also ask the AI to generate code comments/documentation, summaries of those documentation, overview of the system, and re-review them all for correctness for the changes they asked the AI to do. What I've noticed reviewing all my colleagues' AI generated code PRs is: it really is just code, and the rare comment here and there is still added by the human. We're already trying to ligh…

What harness? In my experience, Claude shoots out tons of very low-quality comments. It's always too hyper-focused on the exact specifics of the bug as presented to it, with no higher-level generalization of the concepts involved. In a big codebase, this means the comments are meaningless without a human rewrite, but it definitely writes them.

Re: -​-dangerously-skip-reading-code

#175

Post by tyleo »

skydhash wrote: post #173
dpark wrote: post #94

Earlier quoted context omitted.

My ultimate point is that source code specifies everything the software can or will do, but most of that is not part of the required specification. Most of it is implementation details you don’t care about. The entire reason we have functions and components and modules etc is to isolate engineers from the things we do not need to care about. I should not need to care about the implementation details of most software,…

> I should not need to care about the implementation details of most software, only if it meets my retirements. The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details that have not made it into the specifications. One one side you have the platform and on the other side you have the interface contract (requirements). Saying what’s in the middle does…

Yeah, I'm with you and I'm not seeing much sense in the argument above.

We should only need to specify observable behavior... but observable behavior is so broad that it includes common defects... but also we shouldn't need to specify the lack of defects even though they are observable.

It feels like, "the AI should read my mind and do the right thing without me fully specifying it."

Re: -​-dangerously-skip-reading-code

#176

Post by camgunz »

iloveoof wrote: post #99

Software engineering has always worked this way, just not to ICs. “The LLMs produce non-deterministic output and generate code much faster than we can read it, so we can’t seriously expect to effectively review, understand, and approve every diff anymore. But that doesn’t necessarily mean we stop being rigorous, it could mean we should move rigor elsewhere.“ Direct reports, when delegated tasks by managers, product n…

No, because those direct reports can use tools to build deterministic software. LLMs can't, because they themselves are non-deterministic. They will say they did, and they will be wrong. And the LLM you have check will also say it did, and it will be wrong. Etc etc.

These things just can't be in the critical path. They are ridiculously unreliable.

Re: -​-dangerously-skip-reading-code

#177

Post by EFLKumo »

> just like we don’t read assembly, or bytecode, or transpiled JavaScript

This makes sense since certain higher-level code produces certain lower-level code, while LLM cannot. If the transpired JS code doesn't work we could just find out the bug in minifiers, etc. but one cannot figure out why LLM fails at one task, especially considering LLMs, even SOTA ones, could be strongly affected by even small prompt changes. Taking this into consideration, I don't think this is a sound reasoning why we don't need to review ai-generated code.

> The LLMs produce non-deterministic output and generate code much faster than we can read it, so we can’t seriously expect to effectively review, understand, and approve every diff anymore.

Exactly. However, this could also indicate a weaker review standard instead of just dropping review. We could also suggest an idea where devs mainly review code design or interfaces, leveraging one's *taste*, while leaving strict logic reasoning, validating and testing to other tools or approaches. It cannot pursuade me that the nature of LLM's code generation must lead to a complete cancel of the code review.

Anyway, I'm not opposing this article and its thought of shift in the future is really good.

Re: -​-dangerously-skip-reading-code

#178

Post by alchemism »

StilesCrisis wrote: post #174
kenhwang wrote: post #132

Earlier quoted context omitted.

I always wondered why people don't also ask the AI to generate code comments/documentation, summaries of those documentation, overview of the system, and re-review them all for correctness for the changes they asked the AI to do. What I've noticed reviewing all my colleagues' AI generated code PRs is: it really is just code, and the rare comment here and there is still added by the human. We're already trying to ligh…

What harness? In my experience, Claude shoots out tons of very low-quality comments. It's always too hyper-focused on the exact specifics of the bug as presented to it, with no higher-level generalization of the concepts involved. In a big codebase, this means the comments are meaningless without a human rewrite, but it definitely writes them.

I’ve have amazing, amazing results through composing an “ADR” skill hook. It knows contextually what they are for; how to write them; when to include them as steering.

Re: -​-dangerously-skip-reading-code

#179

Post by demorro »

refactor_master wrote: post #138
throwaw12 wrote: post #59

> my first bet would be specifications and tests You are missing another dimension how easy it would be to migrate if adding new feature hits a ceiling and LLM keeps breaking the system. Imagine all tests are passing and code is confirming the spec, but everything is denormalized because LLM thought this was a nice idea at the beginning since no one mentioned that requirement in the spec. After a while you want to ad…

> Don't forget that very very detailed spec is actually the code The tests, sure. But certainly not the code itself, as that sits far too close to the implementation (i.e. it is the implementation). An almost infinite number of implantations can fulfill “does foo when bar”, so how can we prove that ours is the spec itself? It’s kind of like a scientist coming up with a hypothesis post-hoc to fit the results of the ex…

I know you were simplifying, but "does foo then bar" is so far away from what an actual specification is that it defeats the point.

A more complete spec will capture performance requirements, input preconditions and output postconditions, error handling and recovery behaviors, threading behaviours, hardware assumptions, etc. It's hard to do these things without leaning at least somewhat on the specific language runtime you are using, otherwise you'd end up regurgitating the C standard each time you design a software system.

It's this sort of stuff that is meant when people say "sufficiently detailed".

If you're actually testing all these things, then I might agree with you that you can do it in the tests, but almost no one actually is. I'd struggle to write a test suite that tests all the specification-level assumptions I draw from my language and target platforms.

Re: -​-dangerously-skip-reading-code

#180

Post by trimethylpurine »

EFLKumo wrote: post #177

> just like we don’t read assembly, or bytecode, or transpiled JavaScript This makes sense since certain higher-level code produces certain lower-level code, while LLM cannot. If the transpired JS code doesn't work we could just find out the bug in minifiers, etc. but one cannot figure out why LLM fails at one task, especially considering LLMs, even SOTA ones, could be strongly affected by even small prompt changes.…

Couldn't we slowly add guardrails that eventually lead to code generation becoming more and more deterministic over time?

I'm seeing in my experience that Claude has become better with every version at producing uniformity in its code output. Especially where the architecture is clear and documented. And even more so in languages with built in uniformity (Go, HTMX, SQL) where there is intentionally only one or two ways of doing things. In such environments, the output is nearly deterministic.

Post reply on HN