When AI writes the software, who verifies it? • Hacker News Board

Re: When AI writes the software, who verifies it?

#231

Post by nicbou » Wed, Mar 04, 2026, 9:22 AM UTC

nvlled wrote: post #140

bfung wrote: post #118

When humans write the software, who verifies it? half sarcasm, half real-talk. TDD is nice, but human coders barely do it. At least AI can do it more!

> half sarcasm, half real-talk. If you could pause a bit from being awed by your own perceived insightfulness, you would think a just bit harder and realize that LLMs can generate hundreds of thousands of code that no human could every verify within a finite amount of time. Human-written software is human verifiable, AI-assisted human-written software is still human verifiable to some extent, but purely AI-written so…

That's a very unpleasant tone to take.

Re: When AI writes the software, who verifies it?

#232

Post by scotty79 » Wed, Mar 04, 2026, 9:29 AM UTC

himlion wrote: post #214

scotty79 wrote: post #213

Earlier quoted context omitted.

I wasn't able to force the agent to write failing tests yet. Although I'm sure it should be possible to do.

I do that all the time with Claude. What part is not working?

I don't really use anthropic models. But when I tried it with others they can write tests but they never confirm that they fail before they proceed to make implementation that causes them to pass. Maybe I didn't prompt it forcibly enough.

Re: When AI writes the software, who verifies it?

#233

Post by missingdays » Wed, Mar 04, 2026, 9:40 AM UTC

nly wrote: post #226

mrighele wrote: post #222

Earlier quoted context omitted.

> The LLM happily churns out unit tests which are simply reinforcing the existing behaviour of the code This is true for humans too. Tests should not be written or performed by the same person that writes the code

That's a complete fantasy world where companies have twice the engineers they actually need instead of half.

> [Reviews] should not be written or performed by the same person that writes the code

> That's a complete fantasy world where companies have twice the engineers they actually need instead of half.

Re: When AI writes the software, who verifies it?

#234

Post by JuniperMesos » Wed, Mar 04, 2026, 9:43 AM UTC

muraiki wrote: post #26

The article says that AWS's Cedar authorization policy engine is written in Lean, but it's actually written in Dafny. Writing Dafny is a lot closer to writing "normal" code rather than the proofs you see in Lean. As a non-mathematician I gave up pretty early in the Lean tutorial, while in a recent prototype I learned enough Dafny to be semi-confident in reviewing Claude's Dafny code in about half a day. The Dafny cod…

There's multiple Lean tutorials, some of which are more mathy than others. One of the things I like about Lean is precisely that it's an ordinary, Haskell-style functional programming language in addition to having all the Curry-Howard-isomorphism-based mathematical proof machinery. You can write `cat` in Lean.

Re: When AI writes the software, who verifies it?

#235

Post by KingOfCoders » Wed, Mar 04, 2026, 9:53 AM UTC

Most software the companies I worked for that was put into production, was not verified. There were spotty code reviews, mostly focusing on "I would have done it differently" and a limited amount of unit tests with low test coverage and Heisenberg E2E tests, often turned off because, Heisenberg. Sometimes overworked, bottle neck, testers.

There is hope that with AI we get to better tested, better written, better verified software.

Re: When AI writes the software, who verifies it?

#236

Post by otabdeveloper4 » Wed, Mar 04, 2026, 10:00 AM UTC

Hammershaft wrote: post #129

candiddevmike wrote: post #127

Earlier quoted context omitted.

> and it wasn't long ago before LLMs couldn't do math They still can't do math.

Pro models won gold at the international math olympiads?

[*] According to cloud LLM provider benchmarks.

Re: When AI writes the software, who verifies it?

#237

Post by raincole » Wed, Mar 04, 2026, 10:00 AM UTC

A little mental trick: every time you see a title with AI(or LLM or agent) and a question mark, replace 'AI' with 'humans' and ask the question again.

The answer is no one, for most of the time.

Re: When AI writes the software, who verifies it?

#238

Post by raincole » Wed, Mar 04, 2026, 10:18 AM UTC

fmbb wrote: post #78

madrox wrote: post #42

I encourage everyone to RTFA and not just respond to the headline. This really is a glimpse into where the future is going. I've been saying "the last job to be automated will be QA" and it feels more true every day. It's one thing to be a product engineer in this era. It's another to be working at the level the author is, where code needs to be verifiable. However, once people stop vibing apps and start vibing kerne…

There are still no successful useful vibe codes apps. Kernels are pretty far away I think.

Name 3 apps that are

1. widely considered successful 2. made by humans from scratch in 2025

It looks like humans and AI are on par in this realm.

Re: When AI writes the software, who verifies it?

#239

Post by dustingetz » Wed, Mar 04, 2026, 10:20 AM UTC

WhyNotHugo wrote: post #180

roadbuster wrote: post #139

> The Claude C Compiler illustrates the other side: it optimizes for > passing tests, not for correctness. It hard-codes values to satisfy > the test suite. It will not generalize. This is one of the pain points I am suffering at work: workers ask coding agents to generate some code, and then to generate test coverage for the code. The LLM happily churns out unit tests which are simply reinforcing the existing behavi…

This is why you write the tests first and then the code. Especially when fixing bugs, since you can be sure that the test properly fails when the bug is present.

try this for a UI

Re: When AI writes the software, who verifies it?

#240

Post by wolvesechoes » Wed, Mar 04, 2026, 10:25 AM UTC

KingOfCoders wrote: post #235

Most software the companies I worked for that was put into production, was not verified. There were spotty code reviews, mostly focusing on "I would have done it differently" and a limited amount of unit tests with low test coverage and Heisenberg E2E tests, often turned off because, Heisenberg. Sometimes overworked, bottle neck, testers. There is hope that with AI we get to better tested, better written, better veri…

> There is hope that with AI we get to better tested, better written, better verified software.

And it is one thing we don't get for sure.

This tech, in a different world, could be empowering common people and take some weight from their shoulders. But in this world its purpose is quite the opposite.