Earlier quoted context omitted.
Simple is better than complex Complex is better than complicated Or something, I don't remember...
... simply the best, better than all the rest (Tina Turner)
Ușor, Burebista :-)
Astra for Coding: Why Are We Doing This Again?
221–230 of 334 posts
Re: Astra for Coding: Why Are We Doing This Again?
#222I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…
The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).
> The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).
I understand the reasoning, but at that point wouldn't the LLM be better off creating `sed` commands and executing those? I mean, if it's already executing Python, it can literally do anything to the environment, so using `sed` is at least as safe, with a bonus that it (or a subagent, or a human) can double-check the intention with the sed script and flag incorrect or missing changes.
Re: Astra for Coding: Why Are We Doing This Again?
#223My feeling is that models like Astra and Fable are not made for engineers
Re: Astra for Coding: Why Are We Doing This Again?
#224When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
I'm not a heavy AI user but there are a couple of things I noticed trying to make copilot generate something for me from scratch.
The first is that having a single file with everything you have in mind is very useful. So I end up writing what the project is about, how the model is organized, what each button does, etc. This is good practice in general because writing down everything that the AI will have to consider forces you to consider edge cases before you program them. E.g. if you write "the detail pane shows the fields of the selected item," it makes you consider what should it show when there are no items, or if multiple selection is possible. As you can imagine, this file ends up a very long document even for a simple project because the goal is to pseudo-program everything and let the LLM translate it to an implementation.
Then it still gets things wrong about design, e.g. which pane goes left and which goes right, if you don't also provide an image that shows the layout.
And then, if you supply an exhausting amount of detail, the agent can generate more or less what you had in mind....... or rather, it can generate an OUTPUT that matches your specification from scratch.
The problem is that if there is something you failed to consider, and the AI makes an assumption there, you can end up with a fundamentally broken architecture that you will have to untangle yourself later. And at that point it's easier to write everything from scratch than to fix a pile of AI code that is based on a flawed design.
And it turns out that due to the "totem pole" way that software works, there are infinite places in code that a bad design decision can affect everything it touches.
A good example is how 2 components in a UI are bound to data. You can use events, a bus, state reactivity, etc. Personally I think the mediator pattern is the simplest way to handle GUIs. But an LLM is probably just going to use events for property bindings.
Re: Astra for Coding: Why Are We Doing This Again?
#225Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…
But isn’t “grooming” the hardest part? To know the details means to know the codebases behind. So you do discovery, read code, ask people, check db tables, and then once you figure it out, you write a doc with the proposed solution. It gets reviewed and then you code it.
I definitely need AI help for the discovery part… so it always starts with a simple “I need to do X”
Re: Astra for Coding: Why Are We Doing This Again?
#226When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
I am currently writing a lot of code that I am not reviewing anymore. These are all the things that make it work for me: - Work on small projects ( - Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality. I have never seen a customer or business person care about that. - Have clearly defined API boundaries. Examples: If the back end is…
>> but I look at those same people’s code and it’s horrific, and then I find they haven’t made it far past a proof of concept phase.
> Work on small projects (> Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality.
You then expand on methods and processes that work for you, but I think the crucial question that you do not answer is: How long lived are any of these codebases?
Re: Astra for Coding: Why Are We Doing This Again?
#227When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
Thought expirement. If astra max effort made one function at a time via a heavy description by you. Would it ever produce slop? Fresh context, 1000 lines of style and content guide you personally wrote. Per function no other code context massive guides and styles and it writes a bunch of AI slop unit tests on the side for this one function.
If the answer is yes then atleast you're consistent if no then the question is why can't you scale this until breaks? Then never move beyond that limit?
My argument is there's a "break even" point when the power of the AI is larger than the problem you give it to the point it doesn' slop. You then build at that chunk rate and only try to increase it with next gen model. I usually keep a few "screw it" ideas in my back pocket when a new model arrives to see what happens.
"Go rewrite this entire pipeline in rust" "Go train me a custom x model for y"
Fable is the first model that did not just crash and burn on one of these tasks. Astra still can't do the rust migration (goodbye tokens). But I assume eventually it will. Then I'll have to make up a new ridiculous level.
The model training one was literally an identical pipeline I made before AI and it was like a 6mo process. Fable did it better than me in 1 week (with me helping of course). My theory though is that its datascience is massively higher skill than other systems.
You need to find the chunkrate for your problem and style that works.
Re: Astra for Coding: Why Are We Doing This Again?
#228These machines are doing some crazy things to get to the result. That said, I can't help but feel like this is the compilers argument all over again. Are the methods used to get to the result good? No. Is the code that it generates good? No. Does it achieve the goal. Yes. Is it likely to get better with time. Also yes. In my use cases, jobs that would have taken weeks to months are being done in minutes to hours. Inv…
> But for me, I'm switching to the weaving loom for speed and efficiency.
Poor analogy - the loom was deterministic. LLMs are not, they are probabilistic. I made a page I can point anyone to because I keep seeing this "LLMs are the next level of abstraction" argument.
Re: Astra for Coding: Why Are We Doing This Again?
#229Earlier quoted context omitted.
> I sincerely don’t understand what the people who say they no longer read any code are doing Welcome to the present. Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.
What‘s relevant then? We still put the code in version control, not the shitty prompts that made it.
Correct, but I increasingly find that the version control is just a storage for code and a place to trigger CI/CD from.
Edit: ...and that the source code is useful in retaining enough context of the problem being solved. So, most people will not store the prompts, trusting that the source code provides context for the next iteration.
Re: Astra for Coding: Why Are We Doing This Again?
#230Earlier quoted context omitted.
They're talking about writing a file with a harness-native Edit tool. They're saying the agents aren't doing that, but are using ad-hoc methods of writing the files. (My agents seem to prefer see these days.)
Why do you think your agents prefer to create scripts instead of doing tool calls these days? I wonder, is it easier to modify a script that agent wrote before to satisfy your prompt, or is it easier to write a new one from scratch each time a retry happens? Are input tokens more expensive than output tokens?
My god. They are not reusing the scripts. They are adhoc, inline Python scripts just used to make a single edit. You seem to fundamentally not understand what everyone else is talking about