This article is a great example of how the human is struggling to extrapolate what happens next. There won't be any humans anywhere near this part of the tech stack, just like no one building a SAAS writes assembly code; or has to put together their own server cluster a datacenter (remember pre-cloud?) for their company anymore. He's dead Jim. No one is telling anyone to ship the ML code without any testing. Human coders also make mistakes. I'll bet that in a few years product managers / QA people would rather work with an ML stack to generate code than a human engineering team. It'll not just be cheaper and faster, but a lot less hassle & more accurate. As an example, Python has roughly ~100 or so "keywords", and extensive public libraries and open source algorithms to call upon. Anyone who thinks this presents any sort of challenge for an LLM to profoundly master is is delusional. They can do IMO-grade math, and help proving novel theorems. They can code your YC startup just fine.
The AI coding trap
11–20 of 424 posts
Re: The AI coding trap
#12I do appreciate that this article moves past absolute negativity on LLMs and actually speaks to the fact that they are extremely useful for well defined programming tasks. I'm a bit sick of articles that are just pure negativity on these tools.
I will raise that LLMs are pretty good at some of the non-coding tasks too.
eg. "I'm currently creating an AI for a turn based board game. Without doing any implementation, create a plan for the steps that need to be done including training off real world game data".
The LLM creates a tasklist for iterative steps to accomplish the above. It usually needs correction specific to the business/game needs but it's a great start and i recommend doing this just so the LLM has a doc with context on what its trying to achieve in a bigger picture as you have it complete tasks.
Re: The AI coding trap
#13It's a fine post, but two canards in here: First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning…
> Most of my initial prompts to agents start with "DO NOT WRITE ANY CODE YET." I like asking for the plan of action first, what does it think to do before actually do any edits/file touching.
I’ve also had success writing documentation ahead of time (keeping these in a separate repo as docs), and then referencing it for various stages. The doc will have quasi-code examples of various features, and then I can have a models stubbed in one pass, failing tests in the next, etc.
But there’s a guiding light that both the LLM and I can reference.
Re: The AI coding trap
#14It's a fine post, but two canards in here: First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning…
> the LLM may not be learning anything, but I am
Regardless of that, personally i'd really like it if they could actually learn from interacting with them. From a user's perspective what i'd like to do is to be able to "save" the discussion/session/chat/whatever, with everything the LLM learned so far, to a file. Then later be able to restore it and have the LLM "relearn" whatever is in it. Now, you can already do this with various frontend UIs, but the important part in what i'd want is that a) this "relearn" should not affect the current context window (TBH i'd like that entire concept to be gone but that is another aspect) and b) it should not be some sort of lossy relearning that loses information.
There are some solutions but there are all band-aids to fundamental issues. For example you can occasionally summarize whatever discussed so far and restart the discussion. But obviously that is just some sort of lossy memory compression (i do not care that humans can do the same, LLMs are software running on computers, not humans). Or you could use some sort of RAG but AFAIK this works via "prompt triggering" - i.e. only via your "current" interaction, so even if the knowledge is in there but whatever you are doing now wouldn't trigger its index the LLM will be oblivious to it.
What i want is, e.g., if i tell to the LLM that there is some function `foo` used to barfize moo objects, then go on and tell it other stuff way beyond whatever context length it has, save the discussion or whatever, restore it next day, go on and tell it other stuff, then ask it about joining splarfers, it should be able to tell me that i can join splarfers by converting them to barfized moo objects even if i haven't mentioned anything about moo objects or barfization since my previous session yesterday.
(also as a sidenote, this sort of memory save/load should be explicit since i'd want to be able to start from clean slate - but this sort of clean slate should be because i want to, not as a workaround to the technology's limitations)
Re: The AI coding trap
#15I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless. The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more ti…
"AI" does not encourage real thinking. "AI" encourages hand waving grand plans that don't work, CEO style. All pro-"AI" posts focus on procedures and methodologies, which is just LARPing thinking.
Using "AI" is just like speed reading a math book without ever doing single exercise. The proponents rarely have any serious public code bases.
Re: The AI coding trap
#16It's a fine post, but two canards in here: First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning…
DO NOT WRITE ANY CODE YET.
haha I always do that. I think it's a good way to have some control and understand what it is doing before the regurgitation. I don't like to write code but I love the problem solving/logic/integrations part.
Re: The AI coding trap
#17First time I have to accept cookie profiling on a personal website blog. Also, the cookie banner isn't GDPR-compliant.
Re: The AI coding trap
#18LLM coding agents can't learn from experience on our code, but we can learn from using them on our code, and in the context of our team and processes. I started creating some harnesses to help get more of what we want from these tools, and less of what we need to work too much on - eg, creating specialized agents to refactor code and test after it's been generated, and make it more in line with our standards, removing bogus tests, etc. The learning is embedded in the prompts for these agents.
I think that this approach can already get us pretty far. One thing I'm missing is tooling to make it easier to build automation on top of, eg, Claude Code, but I'm sure it's going to come (and I'm tempted to try vibe coding it; if only I had the time).
Re: The AI coding trap
#19I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless. The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more ti…
My approach has been to "yolo" my way through the first time, yes in a somewhat lazy and careless manner, get a working version, and then build a second time more thoughtfully.
Re: The AI coding trap
#20This article is a great example of how the human is struggling to extrapolate what happens next. There won't be any humans anywhere near this part of the tech stack, just like no one building a SAAS writes assembly code; or has to put together their own server cluster a datacenter (remember pre-cloud?) for their company anymore. He's dead Jim. No one is telling anyone to ship the ML code without any testing. Human co…
Author here - I agree and have written about this before, though focusing a bit more on how far down the stack they might go, rather than up: https://chrisloy.dev/post/2025/03/23/will-ai-replace-softwar...
Article being discussed in this thread isn't intended to be a luddite rejection of AI. It's just a mistake I see people keep making (and have made myself) and some thoughts on how to avoid it with the tools we have today.