Has anybody previously had Gantt chart paths "non-code-1 -> code-1 -> non-code-2 =-> code-2" and transformed them into coding tasks, and taking advantage of the newfound coding speed? What did you do? I would need buy-in from people.
Writing Code Was Never the Bottleneck
41–50 of 400 posts
Re: Writing Code Was Never the Bottleneck
#42My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
> the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be I don't understand, if they don't test the code they write (even if manually) it's not an LLM issue, it's a process one. They have not been taught what does it mean to have a PR ready for being reviewed, LLMs are irrelevant here.
Sometimes, orgs don!t mandate testing or descriptive PRs, and then you requiring it makes you look like a PITA.
Re: Writing Code Was Never the Bottleneck
#43Writing software is like a combination of writing a short story, cleaning your room, and planning a vacation. The bottleneck is always low confidence, much like work anywhere else.
I have watched for almost 20 years employers try to solve and cheat their way around this low confidence. The result is always the same: some shitty form of pattern copy/paste, missing originality, and delivery timelines for really basic features. The reasons for this is that nobody wants to invest in training/baselines and great fear that if they do have something perceived as talent that its irreplaceable and can leave.
My current job in enterprise API management is the first time where the bottleneck is different. Clearly the bottleneck is the customer’s low confidence, as opposed to the developers, and manifests as a very slow requirements gathering process.
Re: Writing Code Was Never the Bottleneck
#44My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
I have a team that’s somewhat junior at a big company. We pretty much have everyone “vibe plan” significantly more than vibe code.
- you need to think through the product more, really be sure it’s as clarified as it can be. Everyone has their own process, but it looks like rubber ducking, critiquing, breaking work into phases, those into tasks, etc. (jobs to be done, business requirement docs, domain driven design planning, UX writing product lexicon docs, literally any and all artifacts)
- Prioritize setting up tooling and feedback loops (code quality tools of any and every kind, are required). this includes custom rules to help enforce anything you decided during planning. Spent time on this and life will be a lot better for everyone.
- We typically making very very detailed plans, and then the agents will “IVI” it (eg automatic linting, single test, test suite, manual evaluation).
You basically set up as many and as diverse of automatic feedback signals as you can.
—-
I will plan and document for 2-4 hours, then print a bunch of small “PRDs” that are like “1 story point” small. There’s clear definitions of done.
Doing this, I can pretty much go the gym or have meetings or whatever for 1-2 hours hands off.
—-
Re: Writing Code Was Never the Bottleneck
#45bottom line is only thing that matters in the end
Re: Writing Code Was Never the Bottleneck
#46My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
> the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be I don't understand, if they don't test the code they write (even if manually) it's not an LLM issue, it's a process one. They have not been taught what does it mean to have a PR ready for being reviewed, LLMs are irrelevant here.
Writing a test requires you to actually know what you're trying to build, and understanding that often requires the slow cooking of a problem that an LLM robs from you. I think this is less of a problem when you've already been thinking deeply on the domain / codebase for a long time. Not true for interns, new hires, interns.
Re: Writing Code Was Never the Bottleneck
#47My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
Well said. That has been my experience as well, but from the perspective of using these tools on my own. Sure, I can now generate thousands of lines of code relatively quickly, but the hard part is actually reviewing the code to ensure that it does what I asked, fix bugs, hunt for security issues, refactor, simplify and remove code, and so on. I've found that it's often much more productive to write the code myself, and rely on the LLM for simple autocomplete tasks on the way. I imagine that this workflow would be much harder when you have to communicate with a less experienced human who will in turn need to translate it to an LLM, because of the additional layers of indirection.
I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether, or they never cared to do them in the first place. Then the burden for maintaining code quality is on the few who actually care, which has now grown much larger because of the amount of code that's thrown at them. Unfortunately, these people are often seen as pedants and sticklers who block PRs for no good reason. That sometimes does happen, but most of the time, these are the folks who actually care about the product shipped to users.
I don't have a suggestion for improving this, but rather a grim outlook that it's only going to get worse. The industry will continue to be flooded by software developers trained on LLM use exclusively, and the companies who build these tools will keep promoting the same marketing BS because it builds hype, and by extension, their valuation.
Re: Writing Code Was Never the Bottleneck
#48I think a lot of teams will wrestle with the existing code review process being abused for quite a while. A lot of people are lazy or get into tech because it’s easy money. The combination of LLMs and a solid code review process means you can submit slop and not even be blamed for the results easier than ever.
Re: Writing Code Was Never the Bottleneck
#49My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
I think this is going to look a lot like the same problem in education, where the answer is that we will have to spend less time consuming written artifacts as a form of evaluation. I think effective code reviews will become more continuous and require much more checking in, asking for explanations as the starting point instead of "I read all of your code and give feedback." That just won't be sustainable given the r…
> instead we should ask people to actually run the code while they explain it to show it working. That's a much better test: show me how it works, and explain it to me.
There’s a reason no one does it. Because it’s inefficient. Even in recorded video format. The helpful things are tests and descriptives PRs. The former because its structure is simple enough that you can judge it, and the test run can be part of the commit. The second is for the simple fact that if you can write clearly about your solution, I can the just do a diff of what you told me and what the code is doing, which is way faster than me trying to divine both from the code.
Re: Writing Code Was Never the Bottleneck
#50My most recent example of this is mentoring young, ambitious, but inexperienced interns. Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before: - During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "go…
> the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be I don't understand, if they don't test the code they write (even if manually) it's not an LLM issue, it's a process one. They have not been taught what does it mean to have a PR ready for being reviewed, LLMs are irrelevant here.
How do you test edge cases?
You think about the implementation and how it can fail. If you don’t think about the implementation, or don’t understand the implementation, I would argue that you can earnestly try to test, but you won’t do a good job of it.
The issue of LLMs here is the proliferation of people not understanding the code they produce.
Having agents or LLMs review and understand and test code may be the future, but right now they’re quite bad at it, and that means that the parent comment is spot on; what I see right now is people producing AI content and pushing the burden of verification and understanding to other people.