Earlier quoted context omitted.
Bullshit.
Not bullshit
Most certainly is.
GPT-6 Astra in code review: Gains, privacy, and cost
71–79 of 79 posts
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#72I ran a few toy benches comparing Astra with Sol and found Astra ~30% faster and at similar cost to Sol for the same outcome https://x.com/__tosh/status/2096201900555170032 the token efficiency helps Astra even though sticker price is 2.5x that of Sol
Looks like a shameless OpenAI plug
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#73Earlier quoted context omitted.
I’m not saying that it’s not useful, I’m saying that it’s not useful in a “human in the loop” situation. This type of AI-to-AI review should be done agent-to-agent, not through Github PRs with tools like Devin. In a manual review, I then expect all “machinery” to already be properly reviewed, and can focus on design / architecture. I would like an AI assisted review tool to make that part easier, not do the actual re…
I’m not convinced by the automatic agent-to-agent thing. I find that, if I manually ask a standard harness “Review a..b, individually and for combined effect”, I get some mix of catching genuine errors (some quite deep), incorrect flags where the correct course of action is to ignore them or modify the commit messages, and comments where the correct course of action is to think deeply. If I were to automate the back…
this touches one of the reasons opus 5 came by default with 200k context and 1M gated behind usage tokens, auto-compaction by default, and it keeps telling you to clear and start from scratch all the time, AI is does a better job fresh, telling multiple agents to pipe around ideas without the human intent is the worse thing you can do
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#74Earlier quoted context omitted.
Well, (AI-generated) test are about half of these PRs' code. So that's still ~8k lines to review... What techno/service did you base your framework on? How long did it take to set it up? How many are you?
That's the point, we don't review the test code either. Our platform gives us a UI for inspecting not the test code but what actually happened during the test. Like a browser replay, the results of a database query, assertions against those, etc. This is much more information dense than something like the tests and is a representation of what actually happened during the tests, rather than what the test itself did (w…
I don't know if that's what you are working on specifically (wink), but there is a product opportunity here.
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#75Earlier quoted context omitted.
That's the point, we don't review the test code either. Our platform gives us a UI for inspecting not the test code but what actually happened during the test. Like a browser replay, the results of a database query, assertions against those, etc. This is much more information dense than something like the tests and is a representation of what actually happened during the tests, rather than what the test itself did (w…
I don't know if that's what you are working on specifically (wink), but there is a product opportunity here.
We are! Got a few pilot customers that are using it but still early days
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#76[flagged]
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#77Earlier quoted context omitted.
What really important things are human reviews catching in your org? I just feel more and more like the effort invested in manual reviews is not worth it
1. Whether the thing should be done in the first place 2. If it's the correct solution on a high level 3. Whether it conflicts with or duplicates other parts of the system 4. Whether the comments are actually useful or restating the LLM chat Also many others but these are the most common IME
I would assume 1/2 are discussed beforehand?
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#78Earlier quoted context omitted.
Yeah I personally don’t understand the point of AI code review tools all that much, as AI is already generating the code as well. All of these AI code review tools create so much noise, yet don’t catch the really important things.
The noise is a huge problem, indeed. Still, a panel of review agents using models and harnesses different from the one implementing a set of changes has proven immensely useful for myself. The panel is basically an n×m matrix of agents and highly specific review prompts, i.e.: - review for intent fulfillment: is the ticket done? - review for correctness: race condition bugs, ... - review for security: check against t…
I bet you beating everyone else by tokens spending in your company
Re: GPT-6 Astra in code review: Gains, privacy, and cost
#79How do you guys review AI-generated code ? In our team, frontend work is vibe-coded by the PO and merged as-is without review. Backend is coded by developers, using AI but in a slower, more controlled way. Recently, our PO has been trying his hand at vibe-coding the backend. I must say he is a smart guy, almost technical but not quite a developer. We've just been handed a burst of stacked PRs amounting for ~15k LOC b…
Code review is soon to be an outmoded concept, (un)fortunately. You have to design orthogonal code (e.g. independent modules in a modular monolith, or microservices) and soak test using canaries.
No code is truly orthogonal if we want it to interact in some way. One microservice might DoS antoher one. In a monolith, some process may take up all resources, and so on.