Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

entelligence.ai

Post reply on HN
Print view View original

61–70 of 123 posts

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#61

Post by dlahoda »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

I would and do.

works perfect

https://github.com/dzmitry-lahoda/dz/tree/main/agents/skills...

burns half of day sub of astra for 200 USD. runs 1 hour on our repo.

finds bugs missed by coderabitai, devin-integration-bot, codex and copilot (I ask them first until nothing found, yet my orchestration finds more).

I do not hardcode our application in orchestration, but run subagent for applicationdomainproduct detection.

also I use agy 200usd sub for second-opinion as one of steps for false positive elimination.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#62

Post by spockz »

jacobgold wrote: post #38
extr wrote: post #31

Earlier quoted context omitted.

I just tell everyone to use Fable 5.1 for everything at this point. Astra is unfortunately a dud, I'm sure they will try to fix a bunch of it with GPT-6.1 but OAI has had this issue for awhile now where every other generation has some sort of strange tic, or reward hacking issue, or something. It's almost like they are balancing the RL on the tip of a needle. Opus 5 has issues too, comment-slop, claude-ish, etc. 5.1…

My trick for using Opus is using it exclusively as a subagent managed by Fable. "Use Opus subagents for this work where possible" is all it takes generally. In my experience Astra/Sol are both quite good as workhorses, but not at Fable's level. I use them every day very successfully and I'm very picky.

Quite good as work horses? To me Luna is the work horse and Sol and Astra are prancing thoroughbreds. If I use Sol or Astra for anything other than curated reasoning and planning I will burn through my usage limits in an hour.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#63

Post by nonethewiser »

stronglikedan wrote: post #52
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

> You wouldn't ask an agent to review a PR then just copy/paste the output into the PR, would you? I would if the PR was clearly written by AI. I'm fine with the PR author using AI, but only to draft the PR. They should be editing the shit out of it for the final version before submitting it.

Workflows:

- Human -> AI: OK

- AI -> AI: OK

- AI -> Human: Not OK (at least here)

AI code reviews are the same as AI pull requests. Do you want that firehose? The suggestions might be good. But do you want to add a deluge of work items to that part of the workflow? Do you want the PR owner to be the one to triage feedback before it gets to them? After they have already done it themselves, perhaps with an even better model?

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#64

Post by heurist »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We totally use it in CI. We were thoughtful in how we built our review bot and it catches a lot of issues before humans lay eyes on the code. Typically they are lower level code issues that humans would not have noticed. It's not the comprehensive review solution, which is why we still require some level of human attendance to the code, but it's very much worth using. We do set limits on PR size and description quality as well to mitigate the harsher AI slop issues.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#65

Post by rockinghigh »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

Human reviews are going away for most PRs these days. If the LLM decides the risk level of the change is low, the PR can be merged without another human in the loop. That's at least the direction many companies are taking. Only require a human judgement when necessary.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#66

Post by nonethewiser »

heurist wrote: post #64
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We totally use it in CI. We were thoughtful in how we built our review bot and it catches a lot of issues before humans lay eyes on the code. Typically they are lower level code issues that humans would not have noticed. It's not the comprehensive review solution, which is why we still require some level of human attendance to the code, but it's very much worth using. We do set limits on PR size and description quali…

Notice you haven't even attempted to address why it's important to have in CI. We already agree AI PR reviews are good. But why should you add this new responsibility for the PR author of triaging feedback?

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#67

Post by dlahoda »

heurist wrote: post #64
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We totally use it in CI. We were thoughtful in how we built our review bot and it catches a lot of issues before humans lay eyes on the code. Typically they are lower level code issues that humans would not have noticed. It's not the comprehensive review solution, which is why we still require some level of human attendance to the code, but it's very much worth using. We do set limits on PR size and description quali…

so I assembled orchestration which catches design issues, multilayer livenesssoundnesssecurity issues.

kind of high level.

did you noticed that astra started to write better comments which look as it understands something?

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#68

Post by aschobel »

jacobgold wrote: post #41
baxtr wrote: post #36

Earlier quoted context omitted.

Could you elaborate on your exact setup? Where do you run these models?

Since you asked, the answer is that I built and use an agent multiplexer called Clor https://clor.com I have a $200/mo Claude subscription and a $200/mo Codex subscription, and I'm signed in to both. The Docker containers keep each session isolated, so dev servers, browser testing, etc. can work without conflicts. It includes `/ask-claude` and `/ask-codex` skills that I use very frequently to have the Claude or Codex…

I have a slightly jankier setup.

Generally using Claude Code with Fable 5.1 (high) to plan and implement (Opus 5 (medium) as the implementer subagents), and using Codex with Astra high to review the plan and review the implementers' output.

Using the OpenAI codoex plugin thingy:

https://github.com/openai/codex-plugin-cc

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#69

Post by heurist »

nonethewiser wrote: post #66
heurist wrote: post #64

Earlier quoted context omitted.

We totally use it in CI. We were thoughtful in how we built our review bot and it catches a lot of issues before humans lay eyes on the code. Typically they are lower level code issues that humans would not have noticed. It's not the comprehensive review solution, which is why we still require some level of human attendance to the code, but it's very much worth using. We do set limits on PR size and description quali…

Notice you haven't even attempted to address why it's important to have in CI. We already agree AI PR reviews are good. But why should you add this new responsibility for the PR author of triaging feedback?

> Typically they are lower level code issues that humans would not have noticed

The author does not triage the feedback. Review bot assesses priority and agents fix the issues the automated review discovers. Nothing is blocked by the bot, humans can ignore if they think the feedback isn't helpful.

We use AI heavily in development but everyone has their own setup and way of approaching use. AI in PRs provides a consistent review layer beyond what the engineers do themselves, and catches a previously undiscovered issue in about 75% of the PRs.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#70

Post by formvoltron »

cheschire wrote: post #48
mchusma wrote: post #39

Earlier quoted context omitted.

We had a two human PR requirement until recently we dropped it. It was slowing us down too much now the human developer creating the future is obviously writing it all with AI so they need to check it then depending on the feature and it’s use it requires a PR but it’s not universal and we’ve stepped up our automated test Tan X what it used to be it’s been so far fewer bugs better delivery

I, too, have noticed a degradation in my grammar when I converse too frequently with AI.

speaking or typing?
Post reply on HN