Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

entelligence.ai

Post reply on HN
Print view View original

41–50 of 123 posts

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#41

Post by jacobgold »

baxtr wrote: post #36
jacobgold wrote: post #8

IMHO, Codex with Astra/Sol and Claude with Fable/Opus are all any professional programmer should be using in Sep 2026, if they can afford it. These models are still terrible compared to what we'd actually wish for, but they're the best available. If you can get away with using the $200/mo subscriptions, it's really not even a money thing for most professionals. Almost all of my work is now plan, generate, review, pla…

Could you elaborate on your exact setup? Where do you run these models?

Since you asked, the answer is that I built and use an agent multiplexer called Clor https://clor.com

I have a $200/mo Claude subscription and a $200/mo Codex subscription, and I'm signed in to both. The Docker containers keep each session isolated, so dev servers, browser testing, etc. can work without conflicts.

It includes `/ask-claude` and `/ask-codex` skills that I use very frequently to have the Claude or Codex harness call out to the other one for advice on plans, bug repro, code review, etc.

The agents run in total "yolo" mode, so there are no permission prompts to approve. The risk is mitigated by the Docker containers (which don't necessarily provide a security barrier but do limit accidents).

I was doing this manually in Ghostty tabs for a long time, and it got painful, so I built a much more sophisticated version that I (and my friends/colleagues) could use.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#42

Post by tomrod »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

Echoing phrasing from the bef-ai-re times: this is the way

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#43

Post by tomrod »

nonethewiser wrote: post #28
VBprogrammer wrote: post #23

Earlier quoted context omitted.

> You wouldn't ask an agent to review a PR then just copy/paste the output int PR would you? Hasn't everyone already got agents directly adding themselves to PRs and leaving comments (occasionally useful)?

They shouldn't. I personally would never dump a claude code response into a PR body an ask someone to address it. I can't believe any developer would find this to be appropriate or fulfilling the duty of reviewing code. I think it's pretty clear that what you should do instead is go over the result and communicate to the PR author anything you think should be addressed. You don't just say "here are 8 things you might…

Indeed.

To add to this, the fact that Claude so often tries to deviate from defined architecture can be really frustrating if you're working in a mature codebase.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#44

Post by nonethewiser »

MattDamonSpace wrote: post #29
VBprogrammer wrote: post #23

Earlier quoted context omitted.

> You wouldn't ask an agent to review a PR then just copy/paste the output int PR would you? Hasn't everyone already got agents directly adding themselves to PRs and leaving comments (occasionally useful)?

Yes. I sympathize with the “every PR needs human eyes on it” ethos but fundamentally the only thing that scales with AI PR generation is AI PR review.

If you are talking about AI's opening PR's then maybe you have a point. But a person opening a PR should not be met with unvetted feedback.

To be absolutely clear, AI should be used for PR review. It should be used many times. By the PR author and by all the reviewers. It should not just be piped directly from the agent to the author inside the PR. That causes the author to triage every comment.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#46

Post by ltbarcly3 »

They seem to conclude Luna is a better value, but their analysis is dumb. They just break it down to $/bug found.

However, Luna missed 23 bugs that Astra found, and identified 24 bugs that weren't really bugs. That's horrible. Astra had 96% precision.

The cost to care about here isn't just how much it costs to run the code review, or the cost per true-positive. It's the cost of dealing with this system. A code review system that is right about 2/3 sucks, and one that misses another 1/3 of the bugs is also a lot worse. The Astra code review quoted here would become the foundation of how the team works, the Luna version is at best helpful to find some stuff but does not dramatically increase your confidence. It also will force humans or better AI's to have to run down a lot of false positives, and that is treated as free here.

Actual conclusion: The cost for Astra is low in absolute terms compared to the cost of bugs and human attention, and the added value is far far more than the added cost.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#47

Post by extr »

triyambakam wrote: post #37
extr wrote: post #31

Earlier quoted context omitted.

I just tell everyone to use Fable 5.1 for everything at this point. Astra is unfortunately a dud, I'm sure they will try to fix a bunch of it with GPT-6.1 but OAI has had this issue for awhile now where every other generation has some sort of strange tic, or reward hacking issue, or something. It's almost like they are balancing the RL on the tip of a needle. Opus 5 has issues too, comment-slop, claude-ish, etc. 5.1…

What have you noticed about Astra? I haven't used Claude models lately so I can't compare but it seems fine compared to 5.6 Sol

- It doesn't write great code.

- Occasionally has strange tics around asking for permission for obvious next-steps, implied actions, etc.

- It's very expensive, both in terms of tokens and % usage on subscription plans.

- Relatedly, effort level is unintuitive. Sometimes it seems like higher effort levels are actually cheaper due to not under-thinking and needing to correct work. But other times they are overkill and send the model into rabbitholes.

That said, it's fantastic as a code-reviewer or "hunter seeker". It's better at finding bugs than Fable and "Get this well articulated task done single-mindedly" is an Astra-shaped task.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#48

Post by cheschire »

mchusma wrote: post #39
nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

We had a two human PR requirement until recently we dropped it. It was slowing us down too much now the human developer creating the future is obviously writing it all with AI so they need to check it then depending on the feature and it’s use it requires a PR but it’s not universal and we’ve stepped up our automated test Tan X what it used to be it’s been so far fewer bugs better delivery

I, too, have noticed a degradation in my grammar when I converse too frequently with AI.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#49

Post by jacobgold »

chrisguilbeau wrote: post #24
jacobgold wrote: post #8

IMHO, Codex with Astra/Sol and Claude with Fable/Opus are all any professional programmer should be using in Sep 2026, if they can afford it. These models are still terrible compared to what we'd actually wish for, but they're the best available. If you can get away with using the $200/mo subscriptions, it's really not even a money thing for most professionals. Almost all of my work is now plan, generate, review, pla…

With everything changing it's great to hear others have the same workflow. I added a snapshot step so I'm doing plan, generate step 1, review, snapshot, generate step 2, review, snapshot... That way I have a chance to diff with the previous iteration and clean up comments, modify skills, etc. also if it bonks on a step I'm one snapshot away from trying again... Is there a place people share their workflows other than…

I wish I knew, I haven't had any place to point people to. I'm going to start sharing this on YouTube since I already spend a few hours each week talking some friend or user through the latest best practices.

Re: GPT-5.6 Luna vs. GPT-6 Astra: Is a $1.20 Model Good Enough for Code Review?

#50

Post by gavinboston »

nonethewiser wrote: post #18

AI should be used for code review but not in CI. You should already have 2+ developers looking at most PRs. And these developers should absolutely use AI. The PR author should use AI. But what you should not do is pipe the AI output directly into the PR and tell the PR author to deal with it. That's adding noise to the PR review process. Everything it says is something the PR author needs to validate as relevant, hel…

In our new world of non-deterministic output (that's why we love LLMs! they say such helpful/agreeable/sometimes wrong stuff!), I think CI won't be sufficient. CI is in the realm of Quality Control; when I build the thing, is it to spec and does it do what I need it to do?

But when the model can shift underneath you, I think it will put pressure on Quality Assurance which is an evergreen task. As a parallel, drug manufacturers don't just test their molecule and manufacturing when they build it, they test it regularly to ensure defects haven't crept in because of some unexpected input to their final output. I think that is similar to how software will evolve.

In my work with LLM-included software, I built a tool that evaluates text output relative to a baseline of what's expected. It helps to ensure things don't drift over time. For example, if a hotel chatbot starts telling guests checkout time is at 11pm instead of 11am, that's a real operational problem and ideally should be caught before it impacts customers.

LLMs introduce new classes of problems/risks that we are just starting to understand and develop the tools to manage.

Post reply on HN