One thing I despise about LLMs is transferring the cognitive load to a machine. It’s just another form of tech debt. And you have repay it pretty fast as the project grows.
Writing Code Was Never the Bottleneck
31–40 of 400 posts
Re: Writing Code Was Never the Bottleneck
#32Funny article, but it seems that the author did not get the "Definition of Done" memo. While... > Writing Code Was Never the Bottleneck ...it was also never the job that needed to get done. We wanted to put well working functionality in the hands of users, in an extendible way (so we could add more features later without too much hassle). If lines of code were the metric of success (like "deal value" is for sales) we…
> We wanted to put well working functionality in the hands of users, in an extendible way (so we could add more features later without too much hassle).
I think the author agrees, and is arguing that LLMs don't help with that.
Re: Writing Code Was Never the Bottleneck
#33my LLM win this year was to give the corporate AI my last year's worth of notes, emails and documents and ask it to write my self review. it did a great job. i'm never writing another one of those stupid bits of psychological torture again
otherwise i'm writing embedded systems. fine, LLM, you hold the scope probe and figure out why that PWM is glitching
Re: Writing Code Was Never the Bottleneck
#34The author puts the BLUF: "The actual bottlenecks were, and still are, code reviews, knowledge transfer through mentoring and pairing, testing, debugging, and the human overhead of coordination and communication."
They're not wrong, but they're missing the point. These bottlenecks can be reduced when there are fewer humans involved.
Somewhat cynically:
code reviews: now sometimes there's just one person involved (reviewing LLM code) instead of two (code author + reviewer)
knowledge transfer: fewer people involved means this is less of an overhead
debugging: no change, yet
coordination and communication: fewer people means less overhead
LLMs shift the workload — they don’t remove it: sure, but shifting workload onto automation reduces the people involved
Understanding code is still the hard part: not much change, yet
Teams still rely on trust and shared context: much easier when there are fewer people involved
... and so on.
"Fewer humans involved" remains a high priority goal for a lot of employers. You can never forget that.
Re: Writing Code Was Never the Bottleneck
#35I dont think the authors comments are without merit. My experience has shown me issues are usually more upfront and after the fact. Either the bottleneck between product organizations and engineering on getting decent requirements to know what to build and engineering teams being unwilling to start until they have every I dotted and T crossed. The backend of the problem is that already most of the code e see written…
In my experience the difficulty in building good software is having a good vision of what the end result should look like and how to get there. If the developer is not savvy about the business case, he cannot have that vision, and all he can do is implement requirements as described by the business, which itself doesn't sufficiently understand technology to build the right path.
I tend to agree. Ideas are cheap and can be easily steered around.
The tricky part is always the action plan: how do we achieve X in steps without blowing budget/time/people/other resources?
Re: Writing Code Was Never the Bottleneck
#36Was anyone claiming it is the bottleneck? Seems like a straw man.
In my experience, writing code can be a bottleneck in the sense that it's pretty easy to end up in a scenario where you're effectively "drudging" through a ton of similar LOC that you can't really optimize in any other way.
GUI libraries are a pretty good example of this; almost all the time, you're probably gonna parse the form fields in the exact same way each time, but due to how GUI libraries work, what ends up happening is that you often write multiple lines of function calls where the only difference really is the key you're using to get the variables. You can't really turn it into a function or something like that either; it's just lines of code that have to be written to make the things work and although it should be really easy to predict what you need to do (just update the variable name and the string used in the function call), it can end up wasting non-marginal time.
LLMs being able to help with this sort of thing I would however more consider to be a failure of IDEs being unable to help with it properly than anything else. This sort of task is rote, easy to predict and should even be autogeneratable. Some IDEs even let you, but it's typically hidden in a menu pretty deep in the interface, needing to be enabled by messing with their ever increasing settings menus (when it probably could just be something it can autodetect by checking the file; y'know, that's the reason why people use IDEs instead of a notepad program); it's as if at some point, IDEs changed from assisting you with making code quicker to write to only really being able to somewhat inspect and lint your codebase unless you spend hours configuring them to do otherwise. I mean, that was in part why Sublime Text and VS Code got their foot in the door, even though they have a much smaller feature list than most traditional IDEs; compared to IDEs they're lightweight (which is pretty crazy since VS Code is an Electron app) and they provide pretty much equivalent features for most people. LLMs can often predict what's going to happen next after you've written two or three of these rote lines, which is a pretty good way to get the boring stuff out of the way.
Is that worth the sheer billions of dollars thrown at AI? Almost certainly not if you look at the entire industry (its a massive bubble waiting to pop), but on the customer fees end, for now the price-to-time-saved ratio for getting rid of that rote work is easily worth it in a corporate environment. (I do expect this to change once the AI bubble pops however.)
Re: Writing Code Was Never the Bottleneck
#37My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
> the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be
I don't understand, if they don't test the code they write (even if manually) it's not an LLM issue, it's a process one.
They have not been taught what does it mean to have a PR ready for being reviewed, LLMs are irrelevant here.
Re: Writing Code Was Never the Bottleneck
#38My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
In the medium term I think you have to shift the work upstream to show that they've put in the labour to actually design the feature or the bug fix.
I think we've always had this mental model which needs to change that senior engineers and product managers scope and design features, IC developers (including juniors for simpler work) implement them, and then senior engineers participate in code review.
Right now I can't see the value in having a junior engineer on the team who is unable to think about how certain features should be designed. The junior engineer who previously spent his time spinning tires trying to understand the codebase and all the new technologies he has to get to grips with should instead spend that time trying to figure out how that feature fits into the big picture, consider edge cases, and then propose a design for the feature.
There are many junior engineers who I wouldn't trust with that kind of work, and honestly I don't think they are employable right now.
In the short term, I think you just need to communicate this additional duty of care to make sure that your pull requests are complete because otherwise there's an asymmetry of workload and judge those interns and juniors on how respectful of that they are.
Re: Writing Code Was Never the Bottleneck
#39My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
I think this is going to look a lot like the same problem in education, where the answer is that we will have to spend less time consuming written artifacts as a form of evaluation. I think effective code reviews will become more continuous and require much more checking in, asking for explanations as the starting point instead of "I read all of your code and give feedback." That just won't be sustainable given the rate at which text can now be output.
AI creates the same problem for hiring too: it generates the appearance of knowledge. The problem you and I have as evaluators of that knowledge is there is no other interface to knowledge than language. In a way this is like the oldest philosophy problem in existence. Socrates spent an inordinate amount of time railing against the sophists, people concerned with language and argument rather than truth. We have his same problem, only now on an industrial scale.
To your point about tests, I think the answer is to not focus on automated tests at first (though of course you should have those eventually), but instead we should ask people to actually run the code while they explain it to show it working. That's a much better test: show me how it works, and explain it to me.
Re: Writing Code Was Never the Bottleneck
#40> The actual bottlenecks were, and still are, code reviews, knowledge transfer through mentoring and pairing, testing, debugging, and the human overhead of coordination and communication. All of this wrapped inside the labyrinth of tickets, planning meetings, and agile rituals.
Most of these only exist because one person cannot code fast enough to produce all the code. If one programmer was fast enough, you would not need a team and then you wouldn't have coordination and communication overhead and so on.