Writing Code Was Never the Bottleneck

They’re not juniors. They’re freshmen

To be fair, there was a pretty dumb CVE (which had already been found and fixed by the time the project made the rounds on HN):

https://github.com/cloudflare/workers-oauth-provider/securit...

You can certainly make the argument that this demonstrates risks of AI.

But I kind of feel like the same bug could very easily have been made by a human coder too, and this is why we have code reviews and security reviews. This exact bug was actually on my list of things to check for in review, I even feel like I remember checking for it, and yet, evidently, I did not, which is pretty embarrassing for me.

> A non-summary PR/MR might lead to the question—“WHY is this code here?"

This is precisely what a (good) commit message should answer.

Commits are historical records, sure, but they can include metadata about the change, which should primarily explain why the change was made, what tradeoffs were made and why, and any other pertinent information.

This is useful not just during the code review process, but for posterity whenever someone needs to understand the codebase, while bisecting, etc. If this information is only in the PR, it won't be easy to reference later.

FWIW I'm not against short summaries in PRs that are exceptionally tricky to understand. The PR description is also useful as a living document for keeping track of pending tasks. But in the majority of cases, commit messages should suffice. This is why GitHub, and I'm sure other forges as well, automatically fill out the PR title and description with the commit information, as long as there's only one commit, which is the ideal scenario. For larger PRs, if it doesn't make sense to create multiple, I usually just say "See the commits for details".

I had this experience even before LLMs, in particular when working with developers who came up in a non-western educational environment. There was a mindset that the only thing that matters is making the boss happy, and in a code review context the reviewer plays the role of boss, so the mindset is "do whatever is required for the boss to stop complaining", not "how can I learn from the knowledge this person is sharing". It's a fundamental difference in how people relate to one another professionally, and I think LLMs have spread this kind of attitude into broader cultural contexts - the devaluation of critical thinking and learning as a necessary part of the job and a more mercenary focus on uncritically churning out whatever the boss asked for.

The doomer perspective would be that people are getting dumber and more complacent and that this will unravel society, but that might not actually be the case if we consider that the mindset already existed in other societies that still thrive. Perhaps the people who never really gave a crap about the quality of their work were right all along? After all, despite the fact most of us are in the top 20% of earners in our countries and easily the top 10% or an even more elite minority globally, end of the day we are still "code peasants" who build whatever our boss told us to build so that an ultra-wealthy investor class can compound their wealth. Why should we waste our time caring about that? Why not get an AI to grind out garbage on our behalf? Why not focus our energies on more rewarding pursuits in our personal lives?

Of course I am playing devil's advocate here, because for me personally being forced to show up for work every day thanks to capitalism and then doing a half-assed job makes me more depressed than trying to excel at something I never wanted to do in the first place. But there is a part of me that understands the mindset and wonders if my life might be easier if I shared it.

Anyway, prior to LLMs I dealt with this phenomenon by reluctantly accepting that most people don't care anywhere near as much about the quality of their work as I do, and that it was hopeless trying to change them. Find the few who do care and prioritize actually-productive knowledge exchanges with them. Drop your standards for people who clearly don't care. If the code doesn't meet your standards but it's still more-or-less functional, just let it go. You might imagine it'll reflect poorly on you, except in reality management doesn't care anyway - the push to AI all the things right now is the "mask off" moment. Every now and then you'll still find a motivated junior who really is passionate about getting better and then being a part of their growth is still rewarding.

In theory this makes sense, but in practice now that LLMs are writing the PR summaries we just have even more slop to wade through to figure out exactly what the change is trying to achieve. I think the slide in this direction already started with exhaustive PR templates that required developers to fill in a whole bunch of fluff just to open their PR. The templates didn't make bad developers good, it just caused them to produce more bad content for review.

My experience with LLM-generated summaries is the same as it was with the templates: many complete them in a way that is entirely self-referential and lacking in context. I don't need a comment or a summary to describe to me exactly the same thing I could have already understood by reading the code. The reason for adding English-language annotations to source code is to explain how a particular change solves a complex business problem, or how it fits into a long-term architectural plan, that sort of thing. But the kinds of people who already did not care about that high level stuff don't have the context to write useful summaries, and LLMs don't either.

The worst thing I've seen recently is when you push for more clarity and context on the reasons behind a change, and then that request gets piped into an LLM. The AI subsequently invents a business problem or architectural goal that in reality doesn't exist and then you get a summary that looks plausible in the abstract, and may even support the code changes it is describing, but it still doesn't link back to anything the team or company is actually trying to achieve, and that costs the reviewer even more time to check. AI proponents might say "well they should have fed the team OKRs and company mission/vision/values into the LLM for context" but then that defeats the point of having the code review in the first place. If the output is performative and not instructive, then the whole process is a waste of time.

I am not sure what the solution is, although I do think that this is not a problem that started with LLMs, it's just an evolution of a challenge we have always faced - how to deal with colleagues who are not really engaged.