Man, I hate when I'm reading a blogpost and then I realize the whole thing is just an ad for a startup.
That's why I first read the comments to decide if it's worth my time.
The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
11–20 of 35 posts
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#12I put in a code reviewer that runs and comments when a pull request is created using Github actions and Microsoft GenAIScript. It's pretty straightforward. The key thing is we have total control over the prompt to fit our repo and devs needs, can make it multi-stage and deterministic using Typescript code or use agents in GenAIScript to open adjacent files for more context. The value we've received is that a dev can look over the review to catch anything they might have missed and make changes all before another dev looks at it. That saves time. I've seen devs open draft pull requests to get preliminary feedback on work in progress. The reviewer script is versioned with the repo. Currently using a mix of gpt-4o and gpt-4o-mini in parts of the script to do smaller tasks.
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#13I put in a code reviewer that runs and comments when a pull request is created using Github actions and Microsoft GenAIScript. It's pretty straightforward. The key thing is we have total control over the prompt to fit our repo and devs needs, can make it multi-stage and deterministic using Typescript code or use agents in GenAIScript to open adjacent files for more context. The value we've received is that a dev can…
How do you make it deterministic?
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#14AI is not good enough yet for anything requiring deep reasoning, mission-critical work, error detection at a human-expert level, or handling unpredictable edge cases. It just talks like it's very smart, and humans apparently have a bias for persuasive communication skills. It's also very fast, which humans also think indicates general intelligence. But it's not, and that's why most LLM tools are author-focused, so th…
>AI is not good enough yet for anything requiring deep reasoning, mission-critical work, error detection at a human-expert level, or handling unpredictable edge cases. Ai is better than humans at all those things. It's not good at those things when the context it needs to look over is more than a few thousand tokens. Rejoice programmer, for your inability to write modular code saved your job.
Apt username for such a bonkers response
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#15Earlier quoted context omitted.
>AI is not good enough yet for anything requiring deep reasoning, mission-critical work, error detection at a human-expert level, or handling unpredictable edge cases. Ai is better than humans at all those things. It's not good at those things when the context it needs to look over is more than a few thousand tokens. Rejoice programmer, for your inability to write modular code saved your job.
Apt username for such a bonkers response
[dead]
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#16AI is not good enough yet for anything requiring deep reasoning, mission-critical work, error detection at a human-expert level, or handling unpredictable edge cases. It just talks like it's very smart, and humans apparently have a bias for persuasive communication skills. It's also very fast, which humans also think indicates general intelligence. But it's not, and that's why most LLM tools are author-focused, so th…
> The way you know fully autonomous driving is nowhere near ready
How do you reconcile this claim with Waymo's dramatically increased rate of expansion these past few years?
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#17I put in a code reviewer that runs and comments when a pull request is created using Github actions and Microsoft GenAIScript. It's pretty straightforward. The key thing is we have total control over the prompt to fit our repo and devs needs, can make it multi-stage and deterministic using Typescript code or use agents in GenAIScript to open adjacent files for more context. The value we've received is that a dev can…
How do you make it deterministic?
Sorry I meant that it's javascript / typescript so we can deterministically orchestrate a series of prompts and shape their output exactly how we'd like. Returning the review as structured output as a JSON object is very helpful for this. If the review result seems bungled, run a judge prompt at the end and tell it to go try again ^_^.
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#18Man, I hate when I'm reading a blogpost and then I realize the whole thing is just an ad for a startup.
I'm curious, why? The startup connection is disclosed in the third sentence so it's not particularly hidden. And at least if it's a real product they're putting their money where their mouth, you can check out their product or reviews from customers to see if their approach is right.
Someone working on AI tooling for code reviews is exactly the right person I'd want to get an opinion from on the space, otherwise it's just opining with no validation.
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#19AI is not good enough yet for anything requiring deep reasoning, mission-critical work, error detection at a human-expert level, or handling unpredictable edge cases. It just talks like it's very smart, and humans apparently have a bias for persuasive communication skills. It's also very fast, which humans also think indicates general intelligence. But it's not, and that's why most LLM tools are author-focused, so th…
> The way you know fully autonomous driving is nowhere near ready How do you reconcile this claim with Waymo's dramatically increased rate of expansion these past few years?
Billions of dollars from Google, basically.
https://www.businessinsider.com/robotaxis-may-mobility-tesla...
High operational costs, low revenue potential, technical difficulties, competitors exiting the space.
Re: The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem
#20Earlier quoted context omitted.
> The way you know fully autonomous driving is nowhere near ready How do you reconcile this claim with Waymo's dramatically increased rate of expansion these past few years?
Billions of dollars from Google, basically. https://www.businessinsider.com/robotaxis-may-mobility-tesla... High operational costs, low revenue potential, technical difficulties, competitors exiting the space.
Sorry, that's goalpost moving.
Just reminding you of your earlier claim:
> AI is not good enough yet for anything requiring deep reasoning, mission-critical work...
Is driving a mission-critical function? Due to its safety critical nature, many would say "yes".
So have you simply pivoted to "oh it does work, but it's not as profitable as it should be"?