Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

entelligence.ai

Post reply on HN
Print view View original

51–60 of 123 posts

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#51

Post by nonethewiser »

solsane wrote: post #34
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

In my experience with the Claude Github integration, I found it to be pretty helpful. It’s had a pretty good success rate of catching bugs before they get to master, and for simple ones I can ask it to fix itself. > you should have 2+ developers looking at most PRs It’d be nice, but usually not the case in my experience. More eyes is better. AI review should not replace human review, it should supplement it. I find m…

Did you miss the part where I agreed AI should review PRs?

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#52

Post by stronglikedan »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

> You wouldn't ask an agent to review a PR then just copy/paste the output into the PR, would you?

I would if the PR was clearly written by AI. I'm fine with the PR author using AI, but only to draft the PR. They should be editing the shit out of it for the final version before submitting it.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#55

Post by nonethewiser »

gavinboston wrote: post #50
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

In our new world of non-deterministic output (that's why we love LLMs! they say such helpful/agreeable/sometimes wrong stuff!), I think CI won't be sufficient. CI is in the realm of Quality Control; when I build the thing, is it to spec and does it do what I need it to do? But when the model can shift underneath you, I think it will put pressure on Quality Assurance which is an evergreen task. As a parallel, drug man…

>But when the model can shift underneath you, I think it will put pressure on Quality Assurance which is an evergreen task. As a parallel, drug manufacturers don't just test their molecule and manufacturing when they build it, they test it regularly to ensure defects haven't crept in because of some unexpected input to their final output. I think that is similar to how software will evolve.

The instinct seems good because it's impossible to keep up with all the details if you are running AI full-blast. Absolutely impossible. So testing outputs makes sense.

I have a hard time seeing exactly how we get from here to there. But intuitively I would not be surprised. One of these thing where quality may drop 20% but you can scale 100x.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#56

Post by orphereus »

mchusma wrote: post #39
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We had a two human PR requirement until recently we dropped it. It was slowing us down too much now the human developer creating the future is obviously writing it all with AI so they need to check it then depending on the feature and it’s use it requires a PR but it’s not universal and we’ve stepped up our automated test Tan X what it used to be it’s been so far fewer bugs better delivery

Are people getting burned out?

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#57

Post by gregwebs »

They state Luna is good enough, but its accuracy of findings is 74% whereas Astra is 96%. Dealing with false positives is expensive.

I am finding AI doing its own reviews as part of the process to be the key to productivity. I do subagent (fresh context reviews) at multiple stages with well-specified review criteria. It is really expensive to do with OpenAI or Claude API billing. Deepseek or the discounted monthly plans from OpenAI or Claude can be discounted similar to the 28x they state for Luna compared to Astra and you maintain much higher quality.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#59

Post by jawarner »

extr wrote: post #31
jacobgold wrote: post #8

IMHO, Codex with Astra/Sol and Claude with Fable/Opus are all any professional programmer should be using in Sep 2026, if they can afford it. These models are still terrible compared to what we'd actually wish for, but they're the best available. If you can get away with using the $200/mo subscriptions, it's really not even a money thing for most professionals. Almost all of my work is now plan, generate, review, pla…

I just tell everyone to use Fable 5.1 for everything at this point. Astra is unfortunately a dud, I'm sure they will try to fix a bunch of it with GPT-6.1 but OAI has had this issue for awhile now where every other generation has some sort of strange tic, or reward hacking issue, or something. It's almost like they are balancing the RL on the tip of a needle. Opus 5 has issues too, comment-slop, claude-ish, etc. 5.1…

Interesting, in my experience Astra is a marked improvement over both Sol 5.6 and Fable 5.1. Its output feels a lot more natural, and it is just less "dumb." But individual experiences may vary.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#60

Post by samuelknight »

OriginalPenguin wrote: post #32

It would have been extremeley nice if the original article had also included Sol and Terra in the testing. (Also would have been nice if they included the equivalent Anthropic models for comparison as well, but it's not quite as relevant.)

Luna is interesting because OAI dropped the price by 5x. Astra is interesting because it's OAI's frontier model. They are asking a specific question about Luna's usefulness compared to a frontier model. They they answer that question in their article which was straight to the point and not cluttered with information about mid-tier models.
Post reply on HN