31–40 of 52 posts
If it takes 17 rounds of review from 5 different models/harnesses – I don't care. Just spit out the right code the first time. Otherwise I'm wasting my time clicking "review this" over and over until the PR is worth actually having a human look at.
Earlier quoted context omitted.
This cost seems wild. For comparison GitHub Copilot Code Review is four cents per review once you're outside of the credits included with your subscription.
Yea, but copilot review is useless. The noise it generates easily costs that much in wasted time.
> Reviews are billed on token usage and generally average $15–25, scaling with PR size and complexity. You've got to be completely insane to use AI coding tools at this point. This is the subsidised cost to get users to use it, it could trivially end up ten times this amount. Plus, you've got the ultimate perverse incentive where the company that is selling you the model time to create the PRs is also selling you the…
The relevant comparison for most enterprise isn't whether $15/PR is subsidised - it's whether it beats the alternative. For most shops that's cheap offshore labour plus the principal engineer time spent reviewing it, managing it, and fixing what got merged anyway. Most enterprise code is trivial CRUD - if the LLM generates it and reviews it to an equivalent standard, you're already ahead.
> We've been running Code Review internally for months: on large PRs (over 1,000 lines changed), 84% get findings, averaging 7.5 issues. On small PRs under 50 lines, that drops to 31%, averaging 0.5 issues. Engineers largely agree with what it surfaces: less than 1% of findings are marked incorrect. So the take would be that 84% heavily Claude driven PRs are riddled with ~7.5 issues worthy bugs. Not a great ad of age…
How many bugs do a human introduce in 1000 line PRs and 50 line PRs?
> Reviews are billed on token usage and generally average $15–25, scaling with PR size and complexity. You've got to be completely insane to use AI coding tools at this point. This is the subsidised cost to get users to use it, it could trivially end up ten times this amount. Plus, you've got the ultimate perverse incentive where the company that is selling you the model time to create the PRs is also selling you the…
The concept of "AI will review AI-authored PRs" seems completely wrong to me. Why didn't the AI write the correct code in the first place? If it takes 17 rounds of review from 5 different models/harnesses – I don't care. Just spit out the right code the first time. Otherwise I'm wasting my time clicking "review this" over and over until the PR is worth actually having a human look at.
I've had multiple situations where things "just worked" and at other times you just have to steer it in the right direction a few times, having another agent doing the review works really well (with the right guardrails), it's like having someone with no other intent or bias review your code.
Unless you're talking about "vibe coding" in which case "correct" doesn't really matter as you're not even looking at what the output is, just let it go back/forth until something that works comes out, I haven't had much success or even enjoyed it as much working this way, took me a couple of months to find the sweet spot (my sweetspot, I think it'll be different for everyone).
Interesting: "Reviews are billed on token usage and generally average $15–25, scaling with PR size and complexity."
Average _per review_? Insane costs, that's potentially thousands per developer. Am I missing something?
Does AI review of AI generated code even make sense?
So their business model is to deliver me buggy code and then charge me to fix it?
> Reviews are billed on token usage and generally average $15–25, scaling with PR size and complexity. You've got to be completely insane to use AI coding tools at this point. This is the subsidised cost to get users to use it, it could trivially end up ten times this amount. Plus, you've got the ultimate perverse incentive where the company that is selling you the model time to create the PRs is also selling you the…
Nah, you're insane if you totally change your workflow to the point where you're reliant on them and your skills atrophy though.