81–90 of 308 posts
Okay, so how do I set up the sort of agent / feedback loop he is describing? Can someone point me in the direction to do that? So far all I've done is just open up the windsurf IDE. Do I have to set this up from scratch?
Basically every other IDE probably does it too by now.
Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…
The latter category is totally enamored with LLMs, and I can see the appeal: they don't care at all about the quality or maintainability of the project after it's signed off on. As long as it satisfies most of the requirements, the llm slop / spaghetti is the client's problem now.
The former category (like me, and maybe you) see less value from the LLMs. Although I've started seeing PRs from more junior members that are very obviously written by AI (usually huge chunks of changes that appear well structured but as soon as you take a closer look you realize the "cheerleader effect"... it's all AI slop, duplicated code, flat-out wrong with tests modified to pass and so on) I still fail to get any value from them in my own work. But we're slowly getting there, and I presume in the future we'll have much more componentized code precisely for AIs to better digest the individual pieces.
Earlier quoted context omitted.
Why do people enjoy going to the gym? Those weights have already been lifted a thousand times. I enjoy writing code because of the satisfaction that comes from solving a problem, from being able to create a working thing out of my own head, and to hopefully see myself getting better at programming. I could augment my programming abilities with an LLM in the same way you could augment your gym experience with a forkli…
We are talking past each other here. Once I solved an Advent of Code problem, I felt like the problem wasn't general enough, so I solved the more general version as well. I like programming to the point of doing imaginary homework, then writing myself some extra credit and doing that as well. Way too much for my own good . The point is that solving a new problem is interesting. Solving a problem you already know exac…
That isn't typically what my programming tasks at work consist of. A large part of the work is coming up with what exactly needs to be done, given the existing code and constraints imposed by technical and domain circumstances, and iterating over that. Meaning, this intellectual work isn't detached from the existing code, or from constraints imposed by the language, libraries and tooling. Hence an important part of the intellectual challenges are tied to actually developing and integrating the code yourself. Maybe you don't find those interesting, but they aren't problems one "already knows exactly how to solve". The solution, instead, is the result of a discovery and exploration process.
> The answer is a critical chunk of the work for making agents useful is in the training process of the underlying models. The LLMs of 2023 could not drive agents, the LLMs of 2025 are optimized for it. Models have to robustly call the tools they are given and make good use of them. We are only now starting to see frontier models that are good at this. And while our goal is to eventually work entirely with open models, the open models are trailing the frontier models in our tool calling evals. We are confident the story will change in six months, but for now, useful repeated tool calling is a new feature for the underlying models.
So yes, a software engineering agent is a simple for-loop. But it can only be a simple for-loop because the models have been trained really well for tool use.
In my experience Gemini Pro 2.5 was the first to show promise here. Claude Sonnet / Opus 4 are both a jump up in quality here though. Very rare that tool use fails, and even rarer that it can't resolve the issue on the next loop.
I completely agree with the author's comment that code review is half-hearted and mostly broken. With agents, the bottleneck is really in reading code, not writing it. If everyone is just half-heartedly reviewing code, or using it as a soapbox for their individual preferences, using agents will completely fall apart as they can easily introduce serious security issues or performance hits. Let's be honest, many of tho…
Assume we have excellent test coverage -- the AI can write the code and ensure get the feedback for it being secure / fast / etc.
And the AI can help us write the damn tests!
LLMs for code review, rather than code writing/design could be the killer feature. I think that code review has been broken for a while now, but this could be a way forward. Of particular interest would be security, undefined behaviour, basic misuse of features, double checking warnings out of the compiler against the source code to ensure it isn't something more serious, etc. My current use of LLMs is typically via…
Earlier quoted context omitted.
Just to draw a parallel (not to insult this line of thinking in any way): “ Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to _compile_ your code and then reading it, understand it, fixing it, etc” At a certain point you won’t have to read and understand every line of code it writes, you can trust that a “module” you ask it to build works…
> At a certain point you won’t have to read and understand every line of code it writes, you can trust that a “module” you ask it to build works exactly like you’d think it would, with a clearly defined interface to the rest of your handwritten code. "A certain point" is bearing a lot of load in this sentence... you're speculating about super-human capabilities (given that even human code can't be trusted, and we hav…
Earlier quoted context omitted.
Why do people enjoy going to the gym? Those weights have already been lifted a thousand times. I enjoy writing code because of the satisfaction that comes from solving a problem, from being able to create a working thing out of my own head, and to hopefully see myself getting better at programming. I could augment my programming abilities with an LLM in the same way you could augment your gym experience with a forkli…
OK. Be honest. If you had to write an argument parser once a week, would you enjoy it? Or extracting input from a config file? Or setting up a logger?
Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…
It could do this in code. I didn't have to type anywhere near as much and 1.5 sets of eyes were on it. It did a pretty accurate job and the followup pass was better.
This is just an example I had time to type before my morning shower
Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…
I am beginning to love working like this. Plan a design for code. Explain to the LLM the steps to arrive to a solution. Work on reading, understanding, fixing, planing, ect. while the LLM is working on the next section of code. We are working in parallel. Think of it like being a cook in a restaurant. The order comes in. The cook plans the steps to complete the task of preparing all the elements for a dish. The cook…
One caveat I wonder about is how this kind of constant context switching combines with the need to think deeply (and defensively with non humans). My gut says I'd struggle at also being the brain at the end of the day instead of just the director/conductor.
I've actively paired with multiple people at once before because of a time crunch (and with a really solid team). It was, to this day, the most fun AND productive "I" have ever been and what you're pitching aligns somewhat with that. HOWEVER, the two people who were driving the keyboards were substantially better engineers than me (and faster thinkers) so the burden of "is this right" was not on me in the way it is when using LLMs.
I don't have any answers here - I see the vision you're pitching and it's a very very powerful one I hope is or becomes possible for me without it just becoming a way to burn out faster by being responsible for the deep understanding without the time to grok it.