Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My current strategy is to not read any of the code written by my agents

twitter.com

Post reply on HN
Print view View original

31–40 of 65 posts

Re: My current strategy is to not read any of the code written by my agents

#31

Post by vkou »

reliabilityguy wrote: post #8
daaaaaaan wrote: post #6

This is the guy that thinks optionals are too complicated? https://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath...

Interesting read. To his defense: the argument is not that Optionals are too complicated, but that it’s a wrong path for language design — instead programmers have to test their code properly. I like optionals, but I see his point too.

You can both test your code for correctness, while also letting the compiler enforce correctness, and communicate important information to people reading your code.

Re: My current strategy is to not read any of the code written by my agents

#35

Post by wrs »

andai wrote: post #23

I keep posting this but it keeps being relevant. I had an agent implement a feature completely backwards. It wrote a whole bunch of tests proving the correctness of the implementation. All the tests passed. The really interesting thing to me is that formal verification wouldn't have helped there either -- it would have just written a mathematical proof of the correctness of the backwards feature.

A few models ago it was not unusual to find that Claude had written tests whose assertions looked correct, but were so thoroughly mocked out that they ran no real code at all — a Potemkin test.

I haven’t seen it do that in quite a while, but it was an interesting failure mode!

Re: My current strategy is to not read any of the code written by my agents

#36

Post by Kon5ole »

>My current strategy is to not read any of the code written by my agents. That’s the only way I can take advantage of their productivity.

When you start getting good results from agents you soon realize you are the bottleneck.

Automating the verification of the code is the way to go, otherwise it's just not worth it. It takes longer to read and understand code than to write code, so why bother with agents if you are going to manually review it all anyway?

One thing I miss after ditching Copilot (it got too expensive) was how I could trivially ask for features to be written by one model and verified by another. Have Opus write it and GPT or Gemini verify it.

I figured they were entirely separate models and therefore unlikely to hallucinate in the same way, so it gave me a quick sense of confidence.

Currently I use claude code (different models but all variants of the same) so I have them do planning, review of the plan, implementation, review of the implementation, and unit tests. It's fine, but copilot felt easier.

Re: My current strategy is to not read any of the code written by my agents

#37

Post by 2001zhaozhao »

Right now there is about a 95% chance that when I manually review an agent's code I find nothing that needs changing.

I find it plausible that an extra agentic review pass and more testing can bring this number up to the point that one never needs to review code again. AI writes pretty good code nowadays.

(You still need to be diligent and decide the architecture during the planning, and read the gotchas and "things to note" that the agent will spit out at the end of implementation if it had to diverge from the plan.)

Re: My current strategy is to not read any of the code written by my agents

#38

Post by [deleted] »

simonw wrote: post #28
andai wrote: post #23

I keep posting this but it keeps being relevant. I had an agent implement a feature completely backwards. It wrote a whole bunch of tests proving the correctness of the implementation. All the tests passed. The really interesting thing to me is that formal verification wouldn't have helped there either -- it would have just written a mathematical proof of the correctness of the backwards feature.

What do you mean by "completely backwards"?

[deleted]

Re: My current strategy is to not read any of the code written by my agents

#40

Post by KPGv2 »

relativeadv wrote: post #13
reliabilityguy wrote: post #8

Earlier quoted context omitted.

Interesting read. To his defense: the argument is not that Optionals are too complicated, but that it’s a wrong path for language design — instead programmers have to test their code properly. I like optionals, but I see his point too.

Bob Martin has built a career on asking the wrong questions. He is doing the same here as well.

I think Uncle Bob worship by OOP adherents is crazy, but I also think dismissing him this way is also crazy. His work on popularizing methods for writing good OOP is solid.

But at the end of the day, he's a guy with a long career in programming, which makes him significantly better than the median, but it doesn't make him three sigma above.

In this case, he's got a pretty good short- to medium-term argument that if you have a robust testing and verification suite, AI code that passes it all is a terrific outcome.

I'm really not convinced about long-term. Possibly, AI ends up writing even better in the future and we never have to worry about human maintainability.

But it's also possible AI is at/near its limit, and we'll always need human maintainers. In which case, incomprehensible vibe coding might be a problem.

Post reply on HN