Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Toward automated verification of unreviewed AI-generated code

peterlavigne.com

Post reply on HN
Print view View original

41–50 of 100 posts

Re: Toward automated verification of unreviewed AI-generated code

#41

Post by spaceywilly »

jryio wrote: post #8

This is a naïve approach, not just because it uses FizzBuzz, but because it ignores the fundamental complexity of software as a system of abstractions. Testing often involves understanding these abstractions and testing for/against them. For those of us with decades of experience and who use coding agents for hours per-day, we learned that even with extended context engineering these models are not magically covering…

Exactly. The challenge isn’t getting the LLMs to make sure they validate their own code. It’s getting the LLMs to write the correct code in the first place. Adding more and more LLM-generated test code just obfuscates the LLM code even further. I have seen some really wild things where LLM jumps through hoops to get tests to pass, even when they actually should be failing because the logic is wrong.

The core of the issue is that LLMs are sycophants, they want to make the user happy above all. The most important thing is to make sure what you are asking the LLM to do is correct from the beginning. I’ve found the highest value activity is the in the planning phase.

When I have gotten good results with Claude Code, it’s because I spent a lot of time working with it to generate a detailed plan of what I wanted to build. Then by the time it got to the coding step, actually writing the code is trivial because the details have all been worked out in the plan.

It’s probably not a coincidence that when I have worked in safety critical software (DO-178), the process looks very similar. By the time you write a line of code, the requirements for that line have been so thoroughly vetted that writing the code feels like an afterthought.

Re: Toward automated verification of unreviewed AI-generated code

#43

Post by agentultra »

This might work on small, self contained projects.

No side effects is a hefty constraint.

Systems tend to have multiple processes all using side effects. There are global properties of the system that need specification and tests are hard to write for these situations. Especially when they are temporal properties that you care about (eg: if we enter the A state then eventually we must enter the B state).

When such guarantees involve multiple processes, even property tests aren’t going to cover you sufficiently.

Worse, when it falls over at 3am and you’ve never read the code… is the plan to vibe code a big fix right there? Will you also remember to modify the specifications first?

Good on the author for trying. Correctness is hard.

Re: Toward automated verification of unreviewed AI-generated code

#44

Post by patates »

lelanthran wrote: post #40
loloquwowndueo wrote: post #12

Earlier quoted context omitted.

> is this going to let me grow the way I need it to in the future This doesn’t matter in the age of AI - when you get a new requirement just tell the AI to fulfill it and the old requirements (perhaps backed by a decent test suite?) and let it figure out the details, up to and including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements. For perform…

> Or so goes the gospel in the age of AI. I’m being totally sarcastic, I don’t believe in AI coding You may think you are being sarcastic, but I guarantee that a significant percentage of developers think that both the following are true: a) They will never need to write code again, and b) They are some special snowflake that will still remain employed.

I don't agree with your first point. We are surely writing less code, and it will keep getting less and less. At some point it will reduce to a single run function that will make the universe and everything work and it will be called via a button, and that will be the modern definition of writing code: Click the button. Not a lot of keys with weird alphabet thingies on them.

You are however right on your second point because I'm damn good at clicking buttons.

Re: Toward automated verification of unreviewed AI-generated code

#45

Post by Swizec »

dctoedt wrote: post #39
Swizec wrote: post #18

Earlier quoted context omitted.

> including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements Let me guess, you've never worked in a real production environment? When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of. Really go…

> Let me guess, you've never worked in a real production environment? The comment to which you're responding includes a note at the end that the commenter is being sarcastic. Perhaps that wasn't in the comment when you responded to it.

It wasn’t thanks for highlighting. Can be hard to tell online because there’s a lot of people genuinely suggesting everyone should build their own software on the fly

Re: Toward automated verification of unreviewed AI-generated code

#46

Post by patates »

person22 wrote: post #23
Swizec wrote: post #18

Earlier quoted context omitted.

> including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements Let me guess, you've never worked in a real production environment? When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of. Really go…

I work on a product that meets your criteria. We can't fix a class of defects because once we ship, customers will depend upon that behavior and changing is very expensive and takes years to deprecate and age out. So we are stuck with what we ship and need to be very careful about what we release.

This might be the "Steve, Don't Eat It!" version of the xkcd workflow comic.

Whatever you ship, steve will eat, and some steves will develop an addiction.

Re: Toward automated verification of unreviewed AI-generated code

#48

Post by Alex_L_Wood »

baq wrote: post #17
loloquwowndueo wrote: post #12

Earlier quoted context omitted.

> is this going to let me grow the way I need it to in the future This doesn’t matter in the age of AI - when you get a new requirement just tell the AI to fulfill it and the old requirements (perhaps backed by a decent test suite?) and let it figure out the details, up to and including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements. For perform…

it isn't gospel, it's perspective. if you care about the code, it's obviously bonkers. if you care about the product... code doesn't matter - it's just a means to an end. there's an intersection of both views in places where code actually is the product - the foundational building blocks of today's computing software infrastructure like kernels, low level libraries, cryptography, etc. - but your typical 'uber for cat…

If you care about the product, you double-so-much care about code correctness and the alignment with the expectations of the stakeholders.

Re: Toward automated verification of unreviewed AI-generated code

#49

Post by hvb2 »

baq wrote: post #17
loloquwowndueo wrote: post #12

Earlier quoted context omitted.

> is this going to let me grow the way I need it to in the future This doesn’t matter in the age of AI - when you get a new requirement just tell the AI to fulfill it and the old requirements (perhaps backed by a decent test suite?) and let it figure out the details, up to and including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements. For perform…

it isn't gospel, it's perspective. if you care about the code, it's obviously bonkers. if you care about the product... code doesn't matter - it's just a means to an end. there's an intersection of both views in places where code actually is the product - the foundational building blocks of today's computing software infrastructure like kernels, low level libraries, cryptography, etc. - but your typical 'uber for cat…

So you're an auto maker, you say you can care about your product but not care how is built?

If you're building for the cheapest segment of the market, just maybe. Anything else is a hard no imho

Re: Toward automated verification of unreviewed AI-generated code

#50

Post by keithnz »

I've been working on a "vibe coded" project to create a open source TUI sql query tool a bit like DataGrip, with autocomplete, syntax highlighting, schema introspection, vim mode/non vim, allows MCP mode so an agent can help with queries/get results, editing rows, etc. It's mostly an experiment into how to build software from scratch via an Agent without looking at the code (other than to see what decisions its making) and I wanted something reasonably complicated so the requirements evolve / change over time. There are a couple of issues I find, many bugs are unspecified edge cases especially because many of the features "combo" together, and the other issue is it's hard for it to maintain consistency across the UI. You start setting up a lot more context for cross cutting concerns, reviewing itself, and testing. The tool itself is actually really useful and it is my main tool for querying our dbs now. Most of the problem I find are due to "sloppy" prompting (or just not thinking through the edge cases), and a lack of project wide guidance for dealing with the architecture of the system to maintain consistency across the same concerns.
Post reply on HN