11–20 of 23 posts
Rory Sutherland calls this the "doorman fallacy". We look at a doorman and might naively think their function is to open doors so if we can replace their function, we can replace the person. Then we build an automation, fire the doorman and subsequently discover the doorman was responsible for a multitude of social tasks, like taking in the mail, co-ordinating services, providing small tasks for the residents etc. an…
This obviously is not a replacement for another human looking at your code, and I would not do it in safety critical environments, but it really helps especially in small teams where time is precious and you ship fast.
My only issue is that I would love a dedicated UI where I get this review BEFORE another human looks at the code, so their feedback is not drowned by the AI noise
This well-thought article restricts the notion of code review to pull request assessment - diff file - on an existing code base, perhaps a mostly human-written legacy code base. As a Java code generation researcher, I would appreciate any constructive comments regarding the automated code review of automated changes made to an entirely generated code base, whose live specifications and requirements have been modified…
It links to a Forbes article arguing that the best chess is played by humans and computers working together.
This concept is evidently false. Alpha Zero played super human chess without any human assistance beyond coding its machine learning algorithm and inputting the rules of chess.
Furthermore, in a timed chess match the delay and possible misjudgments from a human would detract from the performance of the human-computer team versus the computer chess program alone.
I can imagine a future where even if an AI code reviewer that autonomously reviewed code resulted in a noticeable drop in some meaningful metrics, like DORA's MTTR or Deployment Frequency, an organization would flip out the very first time the AI made a mistake, even if a human reviewer would have resulted in 10x as many impactful errors.
Rory Sutherland calls this the "doorman fallacy". We look at a doorman and might naively think their function is to open doors so if we can replace their function, we can replace the person. Then we build an automation, fire the doorman and subsequently discover the doorman was responsible for a multitude of social tasks, like taking in the mail, co-ordinating services, providing small tasks for the residents etc. an…
The elevator operator ensures the safety and security of the passengers, makes sure the elevator is properly maintained and assists the elderly and disabled along with numerous other important tasks.
People believe the elevator operator is only responsible for operating the actuating lever and mistakenly believe they can be automated away without fully understanding their full value to the operations of elevators.
This well-thought article restricts the notion of code review to pull request assessment - diff file - on an existing code base, perhaps a mostly human-written legacy code base. As a Java code generation researcher, I would appreciate any constructive comments regarding the automated code review of automated changes made to an entirely generated code base, whose live specifications and requirements have been modified…
Fascinating - are you imagining a sort of adversarial AI situation, where one LLM bot writes the PRs, and a different one reviews them, leading to an organically improving codebase? Kind of a cool idea.
An AGI system that self-improves its code will regenerate every component impacted by the enhancement starting from live system design narratives, useful existing components, relevant design patterns, and intermediate development artifacts that are discarded or become stale in human-driven legacy coding.
I see "agents" as mostly bodies of assembled prompts for LLMs of various strengths used at the appropriate time in the pipeline of code development. A code review agent's prompt would not have the task of generating code and thus not need all that particular context, but would look for historically observed 'gotchas' and flag those for automatic repair, and the repair could go all the way back through the artifact chain to the text requirements and specifications.
Earlier quoted context omitted.
Fascinating - are you imagining a sort of adversarial AI situation, where one LLM bot writes the PRs, and a different one reviews them, leading to an organically improving codebase? Kind of a cool idea.
Pull requests are too-downstream - a legacy I believe. An AGI system that self-improves its code will regenerate every component impacted by the enhancement starting from live system design narratives, useful existing components, relevant design patterns, and intermediate development artifacts that are discarded or become stale in human-driven legacy coding. I see "agents" as mostly bodies of assembled prompts for LL…
Rory Sutherland calls this the "doorman fallacy". We look at a doorman and might naively think their function is to open doors so if we can replace their function, we can replace the person. Then we build an automation, fire the doorman and subsequently discover the doorman was responsible for a multitude of social tasks, like taking in the mail, co-ordinating services, providing small tasks for the residents etc. an…
The truth of this was made clear to me when everyone started doing asynchronous online "code reviews" where the reviewers are just looking at the code (in theory, anyway).
That alone eliminated the majority of the benefit of code reviews, where the author of the code walks through it and explains it in real time. It was amazing how many times I'd seen devs get partway into their explanation then stop when doing that gave them an important insight or revelation that otherwise would never have been surfaced.
Rory Sutherland calls this the "doorman fallacy". We look at a doorman and might naively think their function is to open doors so if we can replace their function, we can replace the person. Then we build an automation, fire the doorman and subsequently discover the doorman was responsible for a multitude of social tasks, like taking in the mail, co-ordinating services, providing small tasks for the residents etc. an…
Probably be more aptly called the "elevator operator fallacy". The elevator operator ensures the safety and security of the passengers, makes sure the elevator is properly maintained and assists the elderly and disabled along with numerous other important tasks. People believe the elevator operator is only responsible for operating the actuating lever and mistakenly believe they can be automated away without fully un…