191–200 of 335 posts
I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…
Sounds like you don't have enough experience with coding agents. Deterministic scripts must always be preferred instead of LLM tool calls. In fact, you should instruct your agents to write code to execute instead of letting them call tools.
Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…
I think this lesson is getting partially outdated. Yes, you need to be specific about what you want, and with earlier LLMs, you need to had both domain knowledge and some general software development experience to front-load various big and small choices about design, architecture and operational reality - what libraries to use, how system components communicate, how you handle auth and store secrets, etc. Otherwise the LLM would pull some random mix of ideas from its latent space, and give you something that's broken in really stupid ways.
Nowadays, it doesn't feel like that to me, not anymore. I still need some understanding to verify the proposals, but I found the last ~6 months of SOTA models to make good choices. Like, just yesterday I asked Claude to design me some simple service, and focused on explaining it the domain parts (nature of systems I want to integrate together, the purpose of that, and the user's priorities and use cases), and the design I got back had specific suggestions around security, authentication, deployment, failover, integration, behavioral impedance-matching between integrated systems, and more, that I all recognized as based on solid software engineering and ops practices, but deviating from it explicitly in every place where it would be wrong for this specific project. The model considered way more corner cases than I did, and I'm actually really impressed by it.
But then, I find greenfield development is easy with LLMs. Modifying existing systems, especially legacy ones, is where I need to babysit and micromanage models - because any misunderstanding or inaccuracy, which often comes from stale documentation or naming mistakes, tends to get amplified and confuse the agents. No matter how precisely you specify your epic, if the model will find something that contradicts your knowledge/intent, there are good chances it'll get confused and make subtle errors, and you won't realize until much later.
The way I see it: models are highly biased to treat everything they read as "ground truth", all of equal importance. There's no nuanced notion that some information may be stale, that there's a temporal and causal order to sources, and that some information may just be wrong.
And this compounds when you let your LLM write code and documentation over time.
When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
For my side project where uptime doesn't really matter but the codebase is quite huge (gamedev of a multiplayer game with Bevy, so both server code, game code, content build pipeline, as well as a TON of editor and authoring code, probably 150k+ LoC), I've been moving towards just reviewing boundaries between modules, APIs, and schemas, and I'm pushing towards keeping things as modular as possible. If something ends…
My comment is more for the people in charge of or working on software teams on complicated products for customers - so many leaders quite a few engineers have utterly drank the koolaid and pushed maximizing AI use with zero regard for quality or even medium term effects. Many of them are getting promoted by other clueless management for it - when someone is handed a huge check for being optimistic, they tend not to second guess themselves.
This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.” My suspicion is that both OpenAI and Anthropic moved their RL agendas from "being rated as useful according to human feedback" to "succeeds at long hor…
They don’t want to sell these tools to developers. They want to cut as many layers as possible.
Developers very rarely blow their limits, except when they're experimenting on purpose.
Most non-developers are out of tokens by the half of the week, and need to use usage credits for the remainder.
To me there is clearly a better target demographic for AI.
I still don’t understand what a “software factory” is. Can someone clue me in?
I don't know how that's supposed to work, but to me it's the most autistic replacement of the actual team that one can come up with.
Earlier quoted context omitted.
Sounds like you don't have enough experience with coding agents. Deterministic scripts must always be preferred instead of LLM tool calls. In fact, you should instruct your agents to write code to execute instead of letting them call tools.
Plenty of experience ;)
When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
- Work on small projects (- Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality. I have never seen a customer or business person care about that.
- Have clearly defined API boundaries. Examples: If the back end is solid you can just vibe code the first version of a front end according to some mockup. Define a data pipeline with steps and clear contracts of what data gets passed around and what each step does. If the LLM messes up one step, rip it out and rewrite it.
- Have clearly set up workflows for tasks. Start with a "ask me everything" phase, then comes a plan phase, a review phase, an implementation phase, another review phase and then the integration phase. Multiple agents going over the same problem catch a surprising amount of dumb stuff that would otherwise slip through.
- Set up excellent testing. Build your whole architecture around being easy to test.
- Have skills for common problems. For one of my projects I had to set up a skill on how to query the ORM, because Claude was consistently doing database operations in a for loop, where it could use batch operations.
The code in the end is better than a lot of the code I've seen humans write.
I don't think this works for everybody and every project. If you have a culture that values code quality and readability, if you work on large existing projects, if you have to nail the architecture of some non-trivial piece of software etc. you are going to have a bad time.
On the other hand you can probably build the MVP of your AI-supported CRUD app 10-20x faster.
I think a lot of the discussions around development tools and techniques just stem from two facts:
1. Developers work on widely different projects with different management and constraints.
2. Tools, libraries, frameworks and code style have to match your mental model, otherwise you going to dislike them.
I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…
The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).
Ask me how I know. Or don't. I have a standing rule for all agents warning about that failure mode (and related, doing `ls` in `/tmp` and few other directories that like to accumulate files by the hundreds..)