Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra for Coding: Why Are We Doing This Again?

lucumr.pocoo.org

Post reply on HN
Print view View original

271–280 of 334 posts

Re: Astra for Coding: Why Are We Doing This Again?

#271

Post by graemep »

taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

I have noticed that when asking LLMs to work with existing code, their output reflects the existing codebase: the better the existing code, the better the LLM code.

Re: Astra for Coding: Why Are We Doing This Again?

#272

Post by rkagerer »

taurath wrote: post #159
dakolli wrote: post #137

Earlier quoted context omitted.

Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process. I'm not convinced this style of "agentic engineering" saves much time. I guess if I was oblivious to what good code looks like, and didn't care about maintainability It wouldn't bother me so much, but it legitimately has e…

Quality code is easy to make changes to. What makes terrible code terrible is that changes become harder and harder to make over time. LLMs haven’t solved this, but the marketing acts as if it has, and many engineering orgs seem to have instantly forgotten the things we’ve learned from the last few decades of building software. Yes you can have your agents do refinement and refactoring. If you don’t know what they’re…

Yeah I expect we're going to see a lot more "disposal code" vomited up.

Re: Astra for Coding: Why Are We Doing This Again?

#273

Post by juvvel »

re-thc wrote: post #241
xgb84j wrote: post #198

Earlier quoted context omitted.

I am currently writing a lot of code that I am not reviewing anymore. These are all the things that make it work for me: - Work on small projects ( - Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality. I have never seen a customer or business person care about that. - Have clearly defined API boundaries. Examples: If the back end is…

> Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality. I have never seen a customer or business person care about that. That's always false. It's like people want their meals delivered fast. They say they don't care about taste or how it is done. Watch when they get sick or don't like it and the drama that happens. People don't care…

This is like the first rule of working with clients -- what the client says they want is often not what they truly want.

Re: Astra for Coding: Why Are We Doing This Again?

#274

Post by skybrian »

API’s and coding standards help a bit. In one case, the AI was testing HTML-generating code with string assertions, so I had it write a test helper that makes a DOM-based testing API available and a skill telling it to write tests that way.

But you need to watch it and intervene when it starts writing code using bad patterns, because it will imitate nearby code.

Re: Astra for Coding: Why Are We Doing This Again?

#275

Post by amai »

The idea to use python code instead of other kinds of tool calls is taken from smolagents: https://huggingface.co/blog/smolagents

It is based on this paper https://huggingface.co/papers/2402.01030 and calls this idea CodeAct. The paper is actually from Apple: https://machinelearning.apple.com/research/codeact

So Astra and Fable seem to take this idea to the extreme causing some unwanted side-effects.

Re: Astra for Coding: Why Are We Doing This Again?

#276

Post by civvv »

gps372 wrote: post #55

Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…

Genuinely curious, what is the benefit of doing this over just programming it yourself? It sounds to me like this is just an extra step that will grant you less control?

When programming a feature, you are simultaneously doing at least four things: 1. Implementing. 2. Building highly detailed mental models. 3. Learning and expanding your skillset. 4. Quality control and scope limiting.

And this process can be iterative and dynamic. Writing massive, super detailed specs that you then hand off to a undeterministic model feels like doing step one and three, while skipping two and four, which you then have to do after. What is the benefit? The speed up, in my opinion, comes if you skip step two and four, but then your product WILL be worse. Feels like I am going crazy?

Programming was never the bottleneck for software dev?

Re: Astra for Coding: Why Are We Doing This Again?

#277

Post by LtWorf »

wallst07 wrote: post #237
sensanaty wrote: post #147

Earlier quoted context omitted.

> I sincerely don’t understand what the people who say they no longer read any code are doing Well that one's easy to answer, they're either A) lying, or B) working on the simplest possible software where this kind of stuff doesn't explode. Or the alternative 3rd option of what you mentioned, the initial pre-MVP phase goes decently but then it all collapses inevitably as the slop accumulates and the codebases become…

Or C, agentic coders who have been doing this for 20+ years (software engineering) know what they're doing? Also, when people say 'read the code' do they really mean go line by line, or review the pseudo code? Meaning, read the high level architecture/data flow. Because IMO the code matters very little if you have the proper testing environment and guardrails, the architecture always matters.

In my circle, the only people who make these claims about AI were all terrible developers whose contributions were slowing down everyone because they needed to go through several reviews and rewrites before they could be accepted.

Now it's basically the same but they love it because it's only their teammates who have to put in the work not them.

This all implodes when the teammates get fed up and just approve everything.

Re: Astra for Coding: Why Are We Doing This Again?

#278

Post by zamadatix »

Two notes from my own poking around to build on with:

5.6 Sol would also run for 20+ hours on prompts with Max or Ultracode. Sometimes this worked out, sometimes it devolved into exactly the nonsense descent into ultra-specific madness seen here. E.g. in one codebase involving physics simulation it, for some reason, spent the last 25% of effort trying to endlessly increase precision. My best guess when reviewing was "at some point it figured the simulation instability was rooted in the accuracy and precision of the numerical approximation in the GPU code, worked really hard on that for a bit, lost the context of the original issue, and got stuck in a deep loop of trying to complete the phase by infinitely working on the numerical accuracy". Perhaps something of a similar nature occurred here.

I've also noticed it's particularly hard to not get Astra to start using scripting languages and the like, particularly over a long horizon. Particularly, I keep getting HTML report artifacts at the end of long implementations even though the projects are typically explicitly set up to just use .md files for any documentation or large summaries. I've even tried steering it away from that in the prompts and agents file for the project, but that the concept of "clean up the fucking build directory when you're done testing" always seem to get left out after a while.

Re: Astra for Coding: Why Are We Doing This Again?

#279

Post by iforgotmypasswo »

taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

I’ll try to explain how to do it correctly. I’m not selling anything. Seeing this as the top comment makes me a bit sad.

  1. Learn about ports and adapters as an architecture pattern. Domain driven design and locality of reasoning are your new best friends. 

  2. Realize that AI can generate unlimited fake data almost immediately. So anything you can isolate can get a fake adapter and a real one. You can build and test any such system in near real time, mounted in some fake data system of your own design.

  3. Give your opaque backend code UI, so you can build it the same way. This can just be a nice log UI that effectively becomes a backend component harness, but you can get fancy now because UI is cheap. Think about the UX here as providing value by making the code maintainable in the field.

  4. Stop thinking like an IC. Don’t be a micromanager about things that don’t matter. Pretend you have 100 mediocre developers working in parallel and design for that explicitly. I actually like go now. It was designed for the managers.

  5. Don’t get lazy. You still have to AI pair program the important bits and make architectural calls. This is actually hard, as you have to prioritize what to review in depth and what to glance over. This is why the backend UI helps. It keeps you in the loop.

  6. The rough model I’m describing scaled decently pre-Astra. Post-Astra is a whole new world because communication and judgement improved. It leaves behind good docs and comments, and explains things clearly. This was the one gap we had with Claude, and it’s fixed now. The code -after several days of testing- is better as well.
On mobile, so I didn’t get super in depth.

Re: Astra for Coding: Why Are We Doing This Again?

#280

Post by prerok »

tripledry wrote: post #202
prerok wrote: post #160

Earlier quoted context omitted.

I think it matters, because nowadays we don't look at assembly any more. I mean, I don't recall the last time I was tracking down a compiler bug but it's definitely been more than 15 years ago. We do have to look at the LLMs' output, though, and, as you already pointed out, iterate to get the correct results. What this means is that the output must still be readable, must be analyzed by someone and I don't see it goi…

I agree from a programmers perspective. But from a broad market and product perspective, for most things you don't need to look at the code. If the product kinda does what it's supposed to. For example, in my game projects I don't look at the CMakeLists anymore, or python scripts that move assets here and there, I can run my game and just see that it did what I expect it to do (renders assets etc). Similar with front…

The trouble, as I see it, is that we still have to look at the code before we run it. Even if it is a one-shot script. And it's just because you never know if there isn't some "rm -rf ${undefined_variable}/" lurking somewhere in there. And if I have to check it then I would very much like it to be readable.

I mean, I do get your point, sometimes it does not matter. Sometimes we could just YOLO it. But... if that then causes a big problem, even if at only 1% of the time, then I don't want to risk it. But that may just be me. YMMV

Post reply on HN