Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

entelligence.ai

Post reply on HN
Print view View original

111–117 of 117 posts

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#111

Post by therealdrag0 »

CharlieDigital wrote: post #81

I found Luna and even 5.4-mini to be quite good at code review provided a few things: 1. Run it in multiple cycles, only on the diff, and only emit a few findings at a time. 2. Give it a memory so each cycle, it knows the previous finding to check if it's been fixed. 3. Give it access to canonical docs that encode your human reviewer heuristics. I exposed these as tool calls so they could be tracked via telemetry. 4.…

Only on the diff? Why? I found AI reviews garbage until they stopped being only on the diff and were actually able to query real context.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#112

Post by mw888 »

gavinboston wrote: post #50
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

In our new world of non-deterministic output (that's why we love LLMs! they say such helpful/agreeable/sometimes wrong stuff!), I think CI won't be sufficient. CI is in the realm of Quality Control; when I build the thing, is it to spec and does it do what I need it to do? But when the model can shift underneath you, I think it will put pressure on Quality Assurance which is an evergreen task. As a parallel, drug man…

> In my work with LLM-included software, I built a tool that evaluates text output relative to a baseline of what's expected. It helps to ensure things don't drift over time.

Is that hotel example real? Curious how exactly you employ this technique—my naive idea was, if talking software development, a sort of 'sanity-check auto-linter agent' catch errors on a regular basis (every 10 seconds, every write, w/e).

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#113

Post by asdfman123 »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

Nah, we have AI code review at Google and it is shockingly good at catching bugs no one would have noticed. I absolutely depend on it now.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#114

Post by mw888 »

nicce wrote: post #97
sixtyj wrote: post #90

Earlier quoted context omitted.

Getting burnout from LLM-assisted coding or any other activity is here. What exhausts and leads to burnout, is probability. Sometimes you get so much dopamine if it goes right. And sometimes you feel like an idiot because LLM does so many mistakes even if you think that you prompt it right. As the pace is so fast, you change your mind’s state so frequently and quickly that it may lead to a (mild) burnout.

> As the pace is so fast, you change your mind’s state so frequently and quickly that it may lead to a (mild) burnout. This is so true. Cognitive brute-force always when dealing with AI.

Certainly not always. There's a hedonic adjustment which happens however, where some tasks go very smoothly without much specification and a lot of "you know what I mean" to the LLM, while others then require you to get painfully specific after it badly misinterprets your intent.

Or maybe you can just get too spoiled with it grokking your intent, then become so vague that your vague ideas are actually just bad ideas. Certainly has happened to me.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#115

Post by tharkun__ »

epolanski wrote: post #85
MattDamonSpace wrote: post #29

Earlier quoted context omitted.

Yes. I sympathize with the “every PR needs human eyes on it” ethos but fundamentally the only thing that scales with AI PR generation is AI PR review.

I don't get the point of AI pr review. If the AI can review the pr, it can do so before the PR exists in the first place. It's a tool/skill/prompt. Just share it with the end user or encode it in the codebase in some files. It may actually be even more useful as that person will use it often and improve it or can it.

That sounds good. But it doesn't work in practice from my experience.

The exact same skill in the hands of one person will have vastly different outcomes in the hands of a different person. The review skill I built myself has been shared with folks. They (say they) use it before they put up a PR. I still catch things with the same skill. That is evolving as I catch the model(s) at BSing. Even if I would re-share it all the time, when I catch it, these other people using the same skill wouldn't catch it.

And that is exactly why building a general purpose review agent (or skill) is harder than me having my own evolving skill. If I have to "properly" release a skill/some automation, I will likely err on the side of not having too many false positives. That is harder than still relying on a human to review the AI reviewer. The same people that were really good actual human reviewers of a PR are probably the ones that are good doing the same thing while AI assisted. And the ones that really needed these other human reviewers and processes to help them be productive are probably going to produce a vibe coded mess if left to their own devices with nothing but some AI.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#116

Post by echelon »

holoduke wrote: post #92
stefangordon wrote: post #86

Earlier quoted context omitted.

Our agents automatically review our PR's - the authors agents automatically see the feedback and make fixes, and automatically merge when everything is green. A well authored CI review process is significantly better than any human could do. We have the AI review not only the changes but clone and investigate all related repositories that integrate with the code to evaluate interactions, check all open and historical…

Must be a token burner setup. Nice if you really got it to work. Now also add business agents that come up with new features based on actual research. Your automated humanless company is almost there.

They're making money, which is more than most ICs living in cost centers can say.

The human code monkey sometimes thinks too highly of the task of coding. That's never once been the job to be done.

Every engineer is going to become a master of understanding and measuring need, opportunity cost, organizational support and headwinds, political will, etc. These are the true features of the job. The coding part will fall away entirely.

Deliver value. Code used to be the expensive part, but now it's becoming cheap, fungible, and ephemeral.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#117

Post by monkey26 »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We’ve added AI to our auto review process. It does expose when the author is not confident in their solution to push back. Which is interesting. But we do try to target specifically at our patterns and safety.
Post reply on HN