The AI coding trap
chrisloy.dev
The AI coding trap
1–10 of 424 posts
Re: The AI coding trap
#2It's a fine post, but two canards in here:
First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning gets recorded and distilled into a design document, something I definitely didn't have the discipline to deliver dependably before LLM agents.
Most of my initial prompts to agents start with "DO NOT WRITE ANY CODE YET."
Second, this idea that LLMs are like junior developers that can't learn anything. First, no they're not. Early-career developers are human beings. LLMs are tools. But the more general argument here is that there's compounding value to working with an early-career developer and there isn't with an LLM. That seems false: the LLM may not be learning anything, but I am. I use these tools much more effectively now than I did 3 months ago. I think we're in the very early stages of figuring how to get good product out of them. That's obvious compounding value.
Re: The AI coding trap
#3I think this would benefit from examples of including coding assistants in the stages enumerated; how can the agent be included in each stage? I've seen posts about successful collaboration with agents at say Google, where there is tons of upfront work among humans to agree on design, then work with the agent to build out parts of the project and ensuring thorough test suites are included.
Does including an agent at each stage of this cycle mean "context engineering"? Is this then just more text and assets to feed in at each stage of LLM ussage to provide the context for the next set of tokens to generate for the next stage of the cycle? Is there something deeper that can be done to encode this level of staged development into the agent's weights/"understanding"? Is there an established process for this yet?
- Specification
- Documentation
- Modular Design
- Test-Driven Development
- Coding Standard
- Monitoring & Introspection
Re: The AI coding trap
#4> While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks: testing to ensure existing functionality isn’t broken, clearing out duplicated code, writing documentation, handling deployment and infrastructure, etc.
I’ve found LLMs just as useful for the "thankless" layers (e.g. tests, docs, deployment).
The real failure mode is letting AI flood the repo with half-baked abstractions without a playbook. It's helpful to have the model review the existing code and plan out the approach before writing any new code.
The leverage may be in using LLMs more systematically across the lifecycle, including the grunt work the author says remains human-only.
Re: The AI coding trap
#5> lack in-depth knowledge of your business, codebase, or roadmap
So give them some context. I like Cline's memory bank approach https://docs.cline.bot/prompting/cline-memory-bank which includes the architecture, progress, road map etc. Some of my more complex projects use 30k tokens just on this, with the memory bank built from existing docs and stuff I told the model along the way. Too much context can make models worse but overall it's a fair tradeoff - it maintains my coding style and architecture decisions pretty well.
I also recommend in each session using Plan mode to get to a design you are happy with before generating any code.
Re: The AI coding trap
#6one axis that is missing from the discussion is how fast they are improving. We need ~35 years to get a senior software engineer (from birth to education to experience). These things are not even 3.5 years old. I am very interested in this space, if you are too dm me on X:@fabmilo I am in SF.
Re: The AI coding trap
#7I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless.
The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more time on the part I personally enjoy, architecting the build and testing the resultant experience.
> While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks
This is, to me, the root of one disagreement I see playing out in every industry where AI has achieved any level of mastery. There's a divide between people who enjoy the physical experience of the work and people who enjoy the mental experience of the work. If the thinking bit is your favorite part, AI allows you to spend nearly all of your time there if you wish, from concept through troubleshooting. But if you like the doing, the typing, fiddling with knobs and configs, etc etc, all AI does is take the good part away.
Re: The AI coding trap
#8It's a fine post, but two canards in here: First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning…
> Most of my initial prompts to agents start with "DO NOT WRITE ANY CODE YET."
I like asking for the plan of action first, what does it think to do before actually do any edits/file touching.
Re: The AI coding trap
#9> Test-Driven Development: generating extensive test cases prior to implementation to guide implementation and prevent regression.
I’ve found this concept trips CC up—- assertions are backwards, confusing comments in the test, etc. Just starting a prompt with “Use TDD to…” really helps.
Re: The AI coding trap
#10I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless. The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more ti…
Most of my anti-AI takes are either:
1) Bad actors using AI at scale to do bad things
2) AI just commodifying everything and making humans into zoo animals