[dead]
Orchestrating AI code review at scale
41–50 of 66 posts
Re: Orchestrating AI code review at scale
#42PR reviews were never the bottleneck
Re: Orchestrating AI code review at scale
#43PR reviews were never the bottleneck
Where I used to sit, they very often were.
Re: Orchestrating AI code review at scale
#44>Code review is a fantastic mechanism for catching bugs and sharing knowledge "Sharing knowledge" is one of the first phrases in the article, and highlighted as a key benefit of code review. But the loss to human-capital from this process is never examined in the post. > Trivial reviews (typo fixes, small doc changes) cost 20 cents on average They did around 25,000 of these runs (about 20% of total). So CF spent $5k…
Well, AI costs are definitely going to go down at least 90% in the next ~18 months for the same quality of output (and probably 90% again in the 24 months after).
Are you sure it's going to make sense to pay someone to do that moving forward?
I don't think it's worth it now, by the way.
It's definitely not going to be worth it in the near future.
Can we even blink for $0.002? What happens when the next 90% increase in efficiency happens??
Re: Orchestrating AI code review at scale
#45PR reviews were never the bottleneck
Well, they are where I am. But LLM reviews doesn't solve that. It just adds another perspective, which catches different issues.
Re: Orchestrating AI code review at scale
#46This blog post is full of small inconsistencies that make it read like a low quality SEO piece. > We also extract a shared context file (shared-mr-context.txt) from the coordinator's prompt and write it to disk. Sub-reviewers read this file instead of having the full MR context duplicated in each of their prompts. This was a deliberate decision, as duplicating even a moderately-sized MR context across seven concurren…
Looks like they need some sort of agent to review their blog posts as well.
Re: Orchestrating AI code review at scale
#47>Code review is a fantastic mechanism for catching bugs and sharing knowledge "Sharing knowledge" is one of the first phrases in the article, and highlighted as a key benefit of code review. But the loss to human-capital from this process is never examined in the post. > Trivial reviews (typo fixes, small doc changes) cost 20 cents on average They did around 25,000 of these runs (about 20% of total). So CF spent $5k…
Well, AI costs are definitely going to go down at least 90% in the next ~18 months for the same quality of output (and probably 90% again in the 24 months after). Are you sure it's going to make sense to pay someone to do that moving forward? I don't think it's worth it now, by the way. It's definitely not going to be worth it in the near future. Can we even blink for $0.002? What happens when the next 90% increase i…
Would like to know where that 90% number comes from, and if it matches historical trend.
Re: Orchestrating AI code review at scale
#48Earlier quoted context omitted.
Well, AI costs are definitely going to go down at least 90% in the next ~18 months for the same quality of output (and probably 90% again in the 24 months after). Are you sure it's going to make sense to pay someone to do that moving forward? I don't think it's worth it now, by the way. It's definitely not going to be worth it in the near future. Can we even blink for $0.002? What happens when the next 90% increase i…
Would like to know where that 90% number comes from, and if it matches historical trend.
https://www.reddit.com/r/LocalLLaMA/comments/1gpr2p4/llms_co...
See Chart 13 here: https://www.rdworldonline.com/ais-great-compression-20-chart...
See here: https://epoch.ai/data-insights/llm-inference-price-trends
LLMs are so comically inefficient compared to the human brain that it is pretty easy to imagine this trend continuing for several more 90% drops.
If LeCun's JEPA or GRAM turn out to be a thing, we could see a 3-4 order of magnitude drop in a single release cycle / generation.
Keep in mind that performance per watt on the hardware side - at the same time - is still doubling every ~24 months - and this doesn't factor that in.
Re: Orchestrating AI code review at scale
#49[flagged]
Re: Orchestrating AI code review at scale
#50Earlier quoted context omitted.
Would like to know where that 90% number comes from, and if it matches historical trend.
https://www.reddit.com/r/LocalLLaMA/comments/1gpr2p4/llms_co... See Chart 13 here: https://www.rdworldonline.com/ais-great-compression-20-chart... See here: https://epoch.ai/data-insights/llm-inference-price-trends LLMs are so comically inefficient compared to the human brain that it is pretty easy to imagine this trend continuing for several more 90% drops. If LeCun's JEPA or GRAM turn out to be a thing, we could se…
[dead]