Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Toward automated verification of unreviewed AI-generated code

peterlavigne.com

Post reply on HN
Print view View original

91–100 of 100 posts

Re: Toward automated verification of unreviewed AI-generated code

#91

Post by skydhash »

SAI_Peregrinus wrote: post #87
hrmtst93837 wrote: post #16

Earlier quoted context omitted.

[flagged]

Or anything where the interaction of small pure functions matters. NAND is a simple pure function. 6 NANDs connected correctly gets you a D flip flop, and suddenly you've got state. Bugs can hide in the combinatorics of all the possible states of your system, and you'll never test them all in polynomial time.

People forget that most tests aren't written to verify if the code is correct or if the spec is being followed. They are mostly written that something isn't broken (mostly to serve as a canary when coding or deploying). There is just too much possible combinations. What you do is doing broad categories and selecting a few candidates to run the code through. But the most important is the theory of the software, having a good understanding of the problem's domain, the model of the solution, and the technical implementation of the solution.

An analogy I've been using is the formula of a curve like y-x^2=0 as the theory of the software. Test points could be (0, 0) (-3, 9), (5, 25). But there's a lot of curves that can pass through these points too. The point's utility is not to prove that you use the correct formula, it's mostly to check if someone has not accidentally change one of the components like the exponent or the minus sign. While the most important for the developer is knowing why we're using this formula.

Re: Toward automated verification of unreviewed AI-generated code

#92

Post by bisonbear »

spaceywilly wrote: post #41
jryio wrote: post #8

This is a naïve approach, not just because it uses FizzBuzz, but because it ignores the fundamental complexity of software as a system of abstractions. Testing often involves understanding these abstractions and testing for/against them. For those of us with decades of experience and who use coding agents for hours per-day, we learned that even with extended context engineering these models are not magically covering…

Exactly. The challenge isn’t getting the LLMs to make sure they validate their own code. It’s getting the LLMs to write the correct code in the first place. Adding more and more LLM-generated test code just obfuscates the LLM code even further. I have seen some really wild things where LLM jumps through hoops to get tests to pass, even when they actually should be failing because the logic is wrong. The core of the i…

I'm becoming convinced that test pass rate is not a great indicator of model quality - instead we have to look at agent behavior beyond the test gate, such as how aligned is it with human intent, and does it follow the repo's coding standards.

I wrote a short blog about this phenomenon here if you're interested https://www.stet.sh/blog/both-pass

also +1 on placing heavy emphasis on the plan. if you have a good plan, then the code becomes trivial. I have started doing a 70/30 or even 80/20 split of time spent on plan / time implementing & reviewing

Re: Toward automated verification of unreviewed AI-generated code

#93

Post by baq »

slopinthebag wrote: post #79
baq wrote: post #54

Earlier quoted context omitted.

Yes? If you’re an auto factory, you might care, but an auto maker cares about minimizing cost and maximizing revenue within the regulatory constraints. Nowhere is there a requirement to care about how the car is built, there are requirements on what the car can and cannot do.

Caring about the requirements on what the car can and cannot do sounds suspiciously like caring about how it's built when you consider how it's built directly impacts what it can and cannot do.

Not al all, it’s what vs how, completely different beasts. How is for engineers to solve, but the business sells the what.

Re: Toward automated verification of unreviewed AI-generated code

#94

Post by anematode »

skydhash wrote: post #91
SAI_Peregrinus wrote: post #87

Earlier quoted context omitted.

Or anything where the interaction of small pure functions matters. NAND is a simple pure function. 6 NANDs connected correctly gets you a D flip flop, and suddenly you've got state. Bugs can hide in the combinatorics of all the possible states of your system, and you'll never test them all in polynomial time.

People forget that most tests aren't written to verify if the code is correct or if the spec is being followed. They are mostly written that something isn't broken (mostly to serve as a canary when coding or deploying). There is just too much possible combinations. What you do is doing broad categories and selecting a few candidates to run the code through. But the most important is the theory of the software, having…

I like this analogy, thanks!

Re: Toward automated verification of unreviewed AI-generated code

#95

Post by fhd2 »

suzzer99 wrote: post #51
fhd2 wrote: post #26

Earlier quoted context omitted.

That's why I find any effort to create specifications... cute. In brownfield software, more often than not, the code _is_ the specification.

But if you start from the beginning with a code base that is always only generated from a spec, presumably as the tools improve you'd be able to grow to a big industrial-grade app that is 100% based on a spec. The question is how many giant apps out there have yet to be even started vs. how many brownfield apps out there that will outlive all of us.

If the spec covers 100% of the code paths, then yes, you're right. But now spec and code are entirely redundant. Changing the spec or changing the code takes the same effort.

If the spec doesn't specify all the details, then there are gaps for the code to fill. For example, code for a UI is highly specific, down to the last pixel. A spec might say "a dialog with two buttons, labelled OK and cancel". That dialog would look different every time the spec is reimplemented.

Unless of course, there was also a spec for the dialog, that we could refer to in the other spec? That's really just code and reuse.

Re: Toward automated verification of unreviewed AI-generated code

#96

Post by maplethorpe »

jryio wrote: post #8

This is a naïve approach, not just because it uses FizzBuzz, but because it ignores the fundamental complexity of software as a system of abstractions. Testing often involves understanding these abstractions and testing for/against them. For those of us with decades of experience and who use coding agents for hours per-day, we learned that even with extended context engineering these models are not magically covering…

Have you tried Claude 4.6 Opus? I think it might be able to do what you're suggesting.

Re: Toward automated verification of unreviewed AI-generated code

#97

Post by maplethorpe »

eggbrain wrote: post #60

I find people over-rotate on whether we should be reviewing AI-produced code. "What if bad code gets into production!" some programmers gasp, as if they themselves have never pushed bad code, or had coworkers do the same. I've worked at places where I've trusted everyone on my team to the extent that most PRs got only a quick glance before getting a "LGTM". On the flipside, I've also worked on teams where every perso…

At least when a human pushed bad code in the past, they could be held accountable.

Re: Toward automated verification of unreviewed AI-generated code

#98

Post by hvb2 »

baq wrote: post #54
hvb2 wrote: post #49

Earlier quoted context omitted.

So you're an auto maker, you say you can care about your product but not care how is built? If you're building for the cheapest segment of the market, just maybe. Anything else is a hard no imho

Yes? If you’re an auto factory, you might care, but an auto maker cares about minimizing cost and maximizing revenue within the regulatory constraints. Nowhere is there a requirement to care about how the car is built, there are requirements on what the car can and cannot do.

Branding is a thing, you know. Especially if you want to sell the high margin cars.

Re: Toward automated verification of unreviewed AI-generated code

#99

Post by SAI_Peregrinus »

skydhash wrote: post #91
SAI_Peregrinus wrote: post #87

Earlier quoted context omitted.

Or anything where the interaction of small pure functions matters. NAND is a simple pure function. 6 NANDs connected correctly gets you a D flip flop, and suddenly you've got state. Bugs can hide in the combinatorics of all the possible states of your system, and you'll never test them all in polynomial time.

People forget that most tests aren't written to verify if the code is correct or if the spec is being followed. They are mostly written that something isn't broken (mostly to serve as a canary when coding or deploying). There is just too much possible combinations. What you do is doing broad categories and selecting a few candidates to run the code through. But the most important is the theory of the software, having…

Yep! I mostly make the point I did to show that "100% coverage" is an impossible metric for any app of even moderate complexity. Regression tests tend to have much more value than trying preemptively find every bug & test for it.

Re: Toward automated verification of unreviewed AI-generated code

#100

Post by eggbrain »

maplethorpe wrote: post #97
eggbrain wrote: post #60

I find people over-rotate on whether we should be reviewing AI-produced code. "What if bad code gets into production!" some programmers gasp, as if they themselves have never pushed bad code, or had coworkers do the same. I've worked at places where I've trusted everyone on my team to the extent that most PRs got only a quick glance before getting a "LGTM". On the flipside, I've also worked on teams where every perso…

At least when a human pushed bad code in the past, they could be held accountable.

There are two opposite answers here, and I feel like I could argue either one:

1) Humans were never held accountable, really

Outside of a few regulated industries, the worst that happens to an engineer who pushes negligent code is that they get fired. But after that happens, what actually changes? The organizational structure of the company that allowed the employee to push bad code still exists.

2) Humans will still be held accountable

If a human (managing a fleet of AI agents, let's say) ends up deploying bad code to production, they won't be able to point to the AI agent and say "it was them that did it!" -- it will still be the human at the end of the line that is held responsible.

Post reply on HN