This one is pretty easy!
Just write your business requirements in a clear, unambiguous and exhaustive manner using a formal specification language.
Bam, no coding required.
Toward automated verification of unreviewed AI-generated code
21–30 of 100 posts
Re: Toward automated verification of unreviewed AI-generated code
#22> The code must pass property-based tests
Who writes the tests? It can be ok to trust code that passes tests if you can trust the tests.
There are, however, other problems. I frequently see agents write code that's functionally correct but that they won't be able to evolve for long. That's also what happened with Anthropic's failed attempt to have agents write a C compiler (not a trivial task, but far from an exceptionally difficult one). They had thousands of good human-written tests, but the agents couldn't get the software to converge. They fixed one bug only to create another.
Re: Toward automated verification of unreviewed AI-generated code
#23Earlier quoted context omitted.
> is this going to let me grow the way I need it to in the future This doesn’t matter in the age of AI - when you get a new requirement just tell the AI to fulfill it and the old requirements (perhaps backed by a decent test suite?) and let it figure out the details, up to and including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements. For perform…
> including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements Let me guess, you've never worked in a real production environment? When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of. Really go…
I work on a product that meets your criteria. We can't fix a class of defects because once we ship, customers will depend upon that behavior and changing is very expensive and takes years to deprecate and age out. So we are stuck with what we ship and need to be very careful about what we release.
Re: Toward automated verification of unreviewed AI-generated code
#24[flagged]
Re: Toward automated verification of unreviewed AI-generated code
#25I think we need to approach provable code.
Re: Toward automated verification of unreviewed AI-generated code
#26Earlier quoted context omitted.
> including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements Let me guess, you've never worked in a real production environment? When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of. Really go…
I work on a product that meets your criteria. We can't fix a class of defects because once we ship, customers will depend upon that behavior and changing is very expensive and takes years to deprecate and age out. So we are stuck with what we ship and need to be very careful about what we release.
That's why I find any effort to create specifications... cute. In brownfield software, more often than not, the code _is_ the specification.
Re: Toward automated verification of unreviewed AI-generated code
#27production ready "fizz buzz" code. lol. I can't even continue typing this response.
Re: Toward automated verification of unreviewed AI-generated code
#28So are we finally past the stage where people pretend they're actually reading any of the code their LLMs are dumping out?
Who's "we"?
I'd consider shipping LLM generated code without review risky. Far riskier than shipping human-generated code without review.
But it's arguably faster in the short run. Also cheaper.
So we have a risk vs speed to market / near term cost situation. Or in other words, a risk vs gain situation.
If you want higher gains, you typically accept more risk. Technically it's a weird decision to ship something that might break, that you don't understand. But depending on the business making that decision, their situation and strategy, it can absolutely make sense.
How to balance revenue, costs and risks is pretty much what companies do. So that's how I think about this kind of stuff. Is it a stupid risk to take for questionable gains in most situations? I'd say so. But it's not my call, and I don't have all the information. I can imagine it making sense for some.
Re: Toward automated verification of unreviewed AI-generated code
#29Earlier quoted context omitted.
> is this going to let me grow the way I need it to in the future This doesn’t matter in the age of AI - when you get a new requirement just tell the AI to fulfill it and the old requirements (perhaps backed by a decent test suite?) and let it figure out the details, up to and including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements. For perform…
> including totally trashing the old implementation and creating an entirely new one from scratch that matches all the requirements Let me guess, you've never worked in a real production environment? When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of. Really go…
> When your software supports 8, 9, 10 or more zeroes of revenue, "trash the old and create new" are just about the scariest words you can say. There's people relying on this code that you've never even heard of.
Well, now it'll take them 5 minutes to rewrite their code to work around your change.
Re: Toward automated verification of unreviewed AI-generated code
#30So are we finally past the stage where people pretend they're actually reading any of the code their LLMs are dumping out?
In a year people will be complaining about human written code going into production without LLM review.