How I program with agents
crawshaw.io
How I program with agents
1–10 of 308 posts
Re: How I program with agents
#2Finally some serious writing about LLMs that doesn’t follow the hype and it faces reality of what can and can’t be useful with these tools.
Really interesting read, although I can’t stand the word “agent” for a for-loop that call recursively an LLM, but this industry is not famous for being sharp with naming things, so here we are.
edit: grammar
Re: How I program with agents
#3Finally some serious writing about LLMs that doesn’t follow the hype and it faces reality of what can and can’t be useful with these tools. Really interesting read, although I can’t stand the word “agent” for a for-loop that call recursively an LLM, but this industry is not famous for being sharp with naming things, so here we are. edit: grammar
It seems like an excellent name, given that people understand it so readily, but what else would you suggest? LoopGPT?
Re: How I program with agents
#4I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs
Re: How I program with agents
#5I completely agree with the author's comment that code review is half-hearted and mostly broken. With agents, the bottleneck is really in reading code, not writing it. If everyone is just half-heartedly reviewing code, or using it as a soapbox for their individual preferences, using agents will completely fall apart as they can easily introduce serious security issues or performance hits.
Let's be honest, many of those can't be found by just 'reading' the code, you have to get your hands dirty and manually debug/or test the assumptions.
Re: How I program with agents
#6I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs
Exactly. Also related on why Natural Language is not really good for programming[0]
[0]: https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...
Anyway I indeed find LLMs useful for stackoverflow-like programming questions. But this seems to not be true for long as SO is dying and updated data on this type of questions will shrink I think.
Re: How I program with agents
#7Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's code lol.
Re: How I program with agents
#8Finally some serious writing about LLMs that doesn’t follow the hype and it faces reality of what can and can’t be useful with these tools. Really interesting read, although I can’t stand the word “agent” for a for-loop that call recursively an LLM, but this industry is not famous for being sharp with naming things, so here we are. edit: grammar
It seems like an excellent name, given that people understand it so readily, but what else would you suggest? LoopGPT?
I’m no better at naming things!
Shall we propose LLM feedback loop systems? It’s more grounded in reality. Agent is like Retina Display to my ears, at least at this stage!
Re: How I program with agents
#9LLMs for code review, rather than code writing/design could be the killer feature. I think that code review has been broken for a while now, but this could be a way forward. Of particular interest would be security, undefined behaviour, basic misuse of features, double checking warnings out of the compiler against the source code to ensure it isn't something more serious, etc.
My current use of LLMs is typically via the search engine when trying to get information about an error. It has maybe a 50% hit rate, which is okay because I'm typically asking about an edge case.
Re: How I program with agents
#10I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs
I like writing code, and it definitely isn't satisfying when an LLM can one-shot a parser that I would have had fun building for hours.
But at the same time, building a parser for hours is also a distraction from my higher level ambitions with the project, and I get to focus on those.
I still get to stub out the types and function signatures I want, but the LLM can fill them in and I move on. More likely I'll even have my go at the implementation but then tag in the LLM when it's not fun anymore.
On the other hand, LLMs have helped me focus on the fun of polishing something. Making sweeping changes are no longer in the realm of "it'd be nice but I can't be bothered". Generating a bunch of tests from examples isn't grueling anymore. Syncing code to the readme isn't annoying anymore. Coming up with refactoring/improvement ideas is easy; just ask and tell it to make the case for you. It has let me be far more ambitious or take a weekend project to a whole new level, and that's fun.
It's actually a software-loving builder's paradise if you can tweak your mindset. You can polish more code, release more projects, tackle more nerdsnipes, and aim much higher. But it took me a while to get over what turned out to be some sort of resentment.