Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI writes the software, who verifies it?

leodemoura.github.io

Post reply on HN
Print view View original

271–280 of 309 posts

Re: When AI writes the software, who verifies it?

#271

Post by Illniyar »

roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

Building a C compiler should not have this problem. There is probably a million test suites coming from outside the LLM that it can sue verify correctness.

Re: When AI writes the software, who verifies it?

#272

Post by GuB-42 »

WhyNotHugo wrote: post #180
roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

This is why you write the tests first and then the code. Especially when fixing bugs, since you can be sure that the test properly fails when the bug is present.

I don't think it addresses the problem.

Writing the tests first and then writing code to pass the tests is no better than writing the code first then writing tests that pass. What matter is that both the code and the tests are written independently, from specs, not from one another.

I think that it is better not to have access to tests when first writing code, as to make sure to code the specs and not code the tests that test the specs as something may be lost in translation. It means that I have a preference for code first, but the ideal case would be for different people to do it in parallel.

Anyway, about AI, in an AI writes both the tests and the code, it will make sure they match no matter what comes first, it may even go back and forth between the tests and code, but it doesn't mean it is correct.

Re: When AI writes the software, who verifies it?

#273

Post by cmrdporcupine »

roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

My only hope is that all of this push leads in the end to the adoption of more formal verification languages and tools.

If people are having to specify things in TLA+ etc -- even with the help of an LLM to write that spec -- they will then have something they can point the LLM at in order for it to verify its output and assumptions.

Re: When AI writes the software, who verifies it?

#274

Post by theshrike79 »

MartyMcBot wrote: post #160
roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

[flagged]

The magic word is "use red/green testing", that makes it create the tests first, confirm they fail (as they should), then it writes the code to match.

Re: When AI writes the software, who verifies it?

#275

Post by theshrike79 »

fmbb wrote: post #78
madrox wrote: post #42

I encourage everyone to RTFA and not just respond to the headline. This really is a glimpse into where the future is going. I've been saying "the last job to be automated will be QA" and it feels more true every day. It's one thing to be a product engineer in this era. It's another to be working at the level the author is, where code needs to be verifiable. However, once people stop vibing apps and start vibing kerne…

There are still no successful useful vibe codes apps. Kernels are pretty far away I think.

Define "successful"?

Does it need to be HN-popular or a household name? Be in the news?

Or something that saves 50% of time by automating inane manual work from a team?

Re: When AI writes the software, who verifies it?

#276

Post by 9rx »

GuB-42 wrote: post #272
WhyNotHugo wrote: post #180

Earlier quoted context omitted.

This is why you write the tests first and then the code. Especially when fixing bugs, since you can be sure that the test properly fails when the bug is present.

I don't think it addresses the problem. Writing the tests first and then writing code to pass the tests is no better than writing the code first then writing tests that pass. What matter is that both the code and the tests are written independently, from specs, not from one another. I think that it is better not to have access to tests when first writing code, as to make sure to code the specs and not code the tests…

Tests are your spec. You write them first because that is the stage when you are still figuring out what you need to write.

Although TDD says that you should only write one test before implementing it, encouraging spec writing to be an iterative process.

Writing the spec after implementation means that you are likely to have forgotten the nuance that went into what you created. That is why specs are written first. Then the nuance is captured up front as it comes to mind.

Re: When AI writes the software, who verifies it?

#278

Post by mattacular »

8note wrote: post #188
roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

> At no point does anyone stop and ask whether the generated code implements the desired functional behaviour for the system ("business logic"). its fun having LLMs because it makes it quite clear that a lot of testing has been cargo-culting. did people ever check often that the tests check for anything meaningful?

Test automation is kind of like a religion. It is comforting to believe that the solution to code is more code.

Re: When AI writes the software, who verifies it?

#279

Post by mycall »

9rx wrote: post #276
GuB-42 wrote: post #272

Earlier quoted context omitted.

I don't think it addresses the problem. Writing the tests first and then writing code to pass the tests is no better than writing the code first then writing tests that pass. What matter is that both the code and the tests are written independently, from specs, not from one another. I think that it is better not to have access to tests when first writing code, as to make sure to code the specs and not code the tests…

Tests are your spec. You write them first because that is the stage when you are still figuring out what you need to write. Although TDD says that you should only write one test before implementing it, encouraging spec writing to be an iterative process. Writing the spec after implementation means that you are likely to have forgotten the nuance that went into what you created. That is why specs are written first. Th…

Also, if you find after implementation that the spec wasn't specific enough, go ahead and refresh the spec and have the LLM redo the code, from scratch if necessary. Writing code is so cheap right now, it takes a different mindset in general.

Re: When AI writes the software, who verifies it?

#280

Post by lenerdenator »

You verify it.

The value proposition of a software engineer is no longer creating code; it is making sure that the ultra-fast code generating capability of the LLM fits the needs of the business.

BDD/TDD, code reviews, deep dives on what the code does, and educating yourself on design patterns... all of these help.

Post reply on HN