Live data from Hacker News

GPT-6 Astra in code review: Gains, privacy, and cost

coderabbit.ai

51–60 of 80 posts

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#51
post #50

Earlier quoted context omitted.

> All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly. No, none of today's AI would give you enough signal around "should this thing be built in the first place" nor if it's the correct solution on a high level. They don't understand why you are doing what you are doing, and even if you explain it, they still don't actually understand the motivation and lots of…

I don’t think I claimed agent reviews to be a panacea. It’s a tool that can help you lower the review pressure in companies working with agentic coding tools.

Someone asked:

> What really important things are human reviews catching in your org?

Another person said:

> 1. Whether the thing should be done in the first place

And you replied:

> All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly

Which as I noted, is very far from the truth. I neither claimed that you said "agent reviews are a panacea", but when you claim "AI can solve all those things" and two of the first items cannot be addressed by AI (today), then I'm rebuking those specific things, not some other general point you implicitly made.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#52

Given that Fable is a Sol-class model, should Astra not be compared to Mythos in those tests?

I heavily A/B tested Opus vs Sol for two weeks, giving Claude Code and Codex the same tasks and comparing the results. In my experience Sol is much closer to Opus than to Fable, with Opus often beating Sol. The only area where Sol is better is code reviews, which these benchmarks confirm. I wish they included Fable.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#53
post #48

Earlier quoted context omitted.

Forking! We run the entire stack (browser, frontend, backend, database, etc) in a Linux VM, so latency between each of the pieces is as tiny as can be. This is quite different from "standard" E2E tests I've seen where the test browsers uses something like a persistent staging environment. The real key is that we can fork that entire Linux VM to take different paths down our testing scenarios, and can run multiple of…

Which VM technology do you use?

[dead]

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#54
post #50

Earlier quoted context omitted.

I don’t think I claimed agent reviews to be a panacea. It’s a tool that can help you lower the review pressure in companies working with agentic coding tools.

Someone asked: > What really important things are human reviews catching in your org? Another person said: > 1. Whether the thing should be done in the first place And you replied: > All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly Which as I noted, is very far from the truth. I neither claimed that you said "agent reviews are a panacea", but when you claim "…

But that’s the thing, when I say "mostly" that sure doesn’t imply it can solve all those things - it can help to a great extent.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#55

Earlier quoted context omitted.

Yeah I personally don’t understand the point of AI code review tools all that much, as AI is already generating the code as well. All of these AI code review tools create so much noise, yet don’t catch the really important things.

Thinking of AI-generated code and AI code reviews as the same "AI" is not correct. The reviewer is using a fresh context window with no previous knowledge of the changes. That is why it's powerful, because you get the agent to interrogate the code without any preconception about the changes. I found the Devin reviewer to be very good, and have heard good things about Cursor's Bugbot. I've also found asking an agent w…

Specialized AI code review software is so pointless though. Back when agents were dumb about git surgery and tool use it might have had a purpose, but now you could replace coderabbit with a skill and I bet the results would be better in some cases.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#56

How do you guys review AI-generated code ? In our team, frontend work is vibe-coded by the PO and merged as-is without review. Backend is coded by developers, using AI but in a slower, more controlled way. Recently, our PO has been trying his hand at vibe-coding the backend. I must say he is a smart guy, almost technical but not quite a developer. We've just been handed a burst of stacked PRs amounting for ~15k LOC b…

Code review is soon to be an outmoded concept, (un)fortunately. You have to design orthogonal code (e.g. independent modules in a modular monolith, or microservices) and soak test using canaries.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#57
post #54

Earlier quoted context omitted.

Someone asked: > What really important things are human reviews catching in your org? Another person said: > 1. Whether the thing should be done in the first place And you replied: > All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly Which as I noted, is very far from the truth. I neither claimed that you said "agent reviews are a panacea", but when you claim "…

But that’s the thing, when I say "mostly" that sure doesn’t imply it can solve all those things - it can help to a great extent.

It is mostly useless at figuring out if "should this thing be built in the first place" and "if it's the correct solution", and mostly cannot help at all with those things.

Where "mostly" means kind of what it says but also not really.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#58
post #18
post #4

Both OAI and Anthropic seem to have released a model that is slightly better but cost ~2x the previous iteration. Interesting play

That likely won’t change if other competitors don’t take the lead at some point. If companies are willing to pay top dollar for the best models AND they get to extract as much money from Chinese labs distilling Astra/Fable it makes no sense to lower prices. Obviously not great for everyday users who don’t have unlimited money.

I don’t think we have data showing that fable is used significantly

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#59
post #36

Earlier quoted context omitted.

Yeah I personally don’t understand the point of AI code review tools all that much, as AI is already generating the code as well. All of these AI code review tools create so much noise, yet don’t catch the really important things.

The noise is a huge problem, indeed. Still, a panel of review agents using models and harnesses different from the one implementing a set of changes has proven immensely useful for myself. The panel is basically an n×m matrix of agents and highly specific review prompts, i.e.: - review for intent fulfillment: is the ticket done? - review for correctness: race condition bugs, ... - review for security: check against t…

You need to literally review the review with another llm pass to push back on the first. Ask it to do something like reassess the severity claims and only surface real P0 to P2 issues.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#60
post #47
post #18

Earlier quoted context omitted.

That likely won’t change if other competitors don’t take the lead at some point. If companies are willing to pay top dollar for the best models AND they get to extract as much money from Chinese labs distilling Astra/Fable it makes no sense to lower prices. Obviously not great for everyday users who don’t have unlimited money.

> If companies are willing to pay top dollar for the best models But they are not willing to pay. https://news.ycombinator.com/item?id=49566137 The referenced NYT article says Open Source LLM's market share increased to 58% from yesteryear's 10%.

(58% on OpenRouter)

I believe both things can be true. OAI/Anthropic selling more tokens than ever, and downloadable weight models increasing in market share.

I doubt OpenAI would copy Fables pricing if it wasn’t financially beneficial. They can only burn through VC funding for so long.

Post reply on HN