Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra in code review: Gains, privacy, and cost

coderabbit.ai

Post reply on HN
Print view View original

21–30 of 80 posts

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#21

Post by 9dev »

jiggawatts wrote: post #13
stingraycharles wrote: post #9

Earlier quoted context omitted.

Yeah I personally don’t understand the point of AI code review tools all that much, as AI is already generating the code as well. All of these AI code review tools create so much noise, yet don’t catch the really important things.

Code review tools are designed for less organised dev teams that don’t do PRs and mandatory human reviews already. It is papering over a lower level of competency without having to invest in actual human oversight or real process improvement.

That's a thoroughly uncharitable view. Especially in smaller orgs with a minimum velocity dictated by the company's need to survive, the amount of code required to be written just to keep up with your competitors is massive. Trying to review that all by hand, thoroughly, is draining, thankless, and tedious. You end up with a few fast movers producing most of the code, and some slower movers forced into a reviewer role they never signed up for. It's an unhealthy dynamic.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#22

Post by Paria_Stark »

9dev wrote: post #19
grokys wrote: post #15

Earlier quoted context omitted.

1. Whether the thing should be done in the first place 2. If it's the correct solution on a high level 3. Whether it conflicts with or duplicates other parts of the system 4. Whether the comments are actually useful or restating the LLM chat Also many others but these are the most common IME

All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly. I also still manually review code, and usually also catch issues, but the severity of what I find shrinks ever further as agents get better. The sprawling code comments are becoming the most draining part of code review though, that's really killing me from the inside.

The AI review are still quite far from having the same level of critical thinking and high level knowledge of your application, what you have done in the past and want to do next etc.

If you don't master this for your own project, what's even the point of your job.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#24

Post by bitlad »

eyalitki wrote: post #7

Comparison was done in the scope of coderabbit AI code review tool, which sadly makes it practically irrelevant. My personal experience as a software engineer, and a former security researcher who did manual code audit, is that this code review tool has such poor results that it isn't worth the "noise" and friction it causes developers during C/I code review

It does add lot of noise after a point you start ignoring the suggestions and findings.

Code generated these days with fable and sol are near perfect. What issues they might have is logical errors.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#25

Post by OtomotO »

bitlad wrote: post #24
eyalitki wrote: post #7

Comparison was done in the scope of coderabbit AI code review tool, which sadly makes it practically irrelevant. My personal experience as a software engineer, and a former security researcher who did manual code audit, is that this code review tool has such poor results that it isn't worth the "noise" and friction it causes developers during C/I code review

It does add lot of noise after a point you start ignoring the suggestions and findings. Code generated these days with fable and sol are near perfect. What issues they might have is logical errors.

> Code generated these days with fable and sol are near perfect.

If you're doing a simple CRUD app, sure.

If you're doing anything more involved they get the job done with dozens of shortcuts that bite you in the ass the moment you have on-call duty.

Way too much code and repetition and hacks.

Especially in GPU code, but also in other fields.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#27

Post by embedding-shape »

9dev wrote: post #19
grokys wrote: post #15

Earlier quoted context omitted.

1. Whether the thing should be done in the first place 2. If it's the correct solution on a high level 3. Whether it conflicts with or duplicates other parts of the system 4. Whether the comments are actually useful or restating the LLM chat Also many others but these are the most common IME

All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly. I also still manually review code, and usually also catch issues, but the severity of what I find shrinks ever further as agents get better. The sprawling code comments are becoming the most draining part of code review though, that's really killing me from the inside.

> All of these are angles an AI reviewer can test for as well, and will (IME) mostly catch mistakes correctly.

No, none of today's AI would give you enough signal around "should this thing be built in the first place" nor if it's the correct solution on a high level.

They don't understand why you are doing what you are doing, and even if you explain it, they still don't actually understand the motivation and lots of other things.

You'll get them to do guesses and pretend they actually know how to prioritize and will tell you it makes lots of sense, whatever they come up with. But try following it blindly and you'll see where you end up.

This is why "one agent + one good developer" beats "thousands of agents working in a swarm" still today.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#28

Post by sdeframond »

How do you guys review AI-generated code ?

In our team, frontend work is vibe-coded by the PO and merged as-is without review. Backend is coded by developers, using AI but in a slower, more controlled way.

Recently, our PO has been trying his hand at vibe-coding the backend. I must say he is a smart guy, almost technical but not quite a developer. We've just been handed a burst of stacked PRs amounting for ~15k LOC backend. We do not quite know what do to about it.

I know we are not the only ones in the situation. What's your experience and context ? What do you do ? What works for you what doesn't ?

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#29

Post by torginus »

ramon156 wrote: post #4

Both OAI and Anthropic seem to have released a model that is slightly better but cost ~2x the previous iteration. Interesting play

Generally speaking 'the Fable/Astra built GTA 6' videos are a new phenomenon, so it's clear these models have new capabilities and people will need new ways of interacting with them if they want to leverage these imo.

Re: GPT-6 Astra in code review: Gains, privacy, and cost

#30

Post by mobelkh »

sdeframond wrote: post #28

How do you guys review AI-generated code ? In our team, frontend work is vibe-coded by the PO and merged as-is without review. Backend is coded by developers, using AI but in a slower, more controlled way. Recently, our PO has been trying his hand at vibe-coding the backend. I must say he is a smart guy, almost technical but not quite a developer. We've just been handed a burst of stacked PRs amounting for ~15k LOC b…

throw his garbage out, the time and effort taken to review that is magnitudes more than what it took to prompt it.

have him start with an overall design doc if his change is 15k, it's definitely worth a design doc.

and then have his contributions reviewed in pieces of 200-300 LoC PRs.

any other solution is trading stability and system knowledge, that's 15k LoC no one is truly familiar with, even if you do try to review it

Post reply on HN