Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra for Coding: Why Are We Doing This Again?

lucumr.pocoo.org

Post reply on HN
Print view View original

211–220 of 334 posts

Re: Astra for Coding: Why Are We Doing This Again?

#211

Post by lelanthran »

dakolli wrote: post #137
taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process. I'm not convinced this style of "agentic engineering" saves much time. I guess if I was oblivious to what good code looks like, and didn't care about maintainability It wouldn't bother me so much, but it legitimately has e…

> Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process.

Where LLMs excel is in code-level bugs (as opposed to system bugs, design bugs, architecture bugs, integration bugs, etc).

Talk with an LLM, ask them to rate both code and estimate dev experience based on that code, and you will see what they value: Code that passes all the tests is a 10/10, while a codebase designed with opaque data types, Parse-Don't-Validate for all data, isolated interfaces, a built-for-replacement-not-for-extension philosphy will get a 6/10 because an out of bounds error was found.

IOW, they are very strongly tuned to value code that has no errors which can be picked up by linters or similar, while humans work the opposite way - we very highly value code that is easy to maintain, even if they do have a few errors picked up by the linter.

Re: Astra for Coding: Why Are We Doing This Again?

#212

Post by ACCount37 »

taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

Enabling more "proof of concept phase" projects to exist is one of the great boons of AI.

If code is expensive, you don't want to commit to a PoC unless you're damn sure. If dirty code is cheap, you can vibe code a PoC early, even if you aren't sure the project is viable. This, of course, leads to more projects dying in PoC phase. It also results in more projects that otherwise wouldn't have gotten to it getting past it.

Personally, I don't believe that "code is shitty and hard make changes in" is in any way, fashion or form an AI-exclusive problem. Big corporations had plenty of decade old codebases filled with decay and rot back in 2009 already. It's just the usual side effect of sacrificing "future maintainability" for "feature velocity" or "expertise" for "cheap labor".

Unlike the usual causes of code rot (cheap replaceable developers, outsourcing to India), AI might actually get out of the pit - by getting good enough at refactoring to be able to beat the code back into shape. There's nothing about refactoring in particular that demands a meatbag when the rest of the coding tasks don't.

Re: Astra for Coding: Why Are We Doing This Again?

#213

Post by casid »

nnevatie wrote: post #209
taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

> I sincerely don’t understand what the people who say they no longer read any code are doing Welcome to the present. Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.

What‘s relevant then? We still put the code in version control, not the shitty prompts that made it.

Re: Astra for Coding: Why Are We Doing This Again?

#214

Post by pazimzadeh »

aytigra wrote: post #177
pazimzadeh wrote: post #124

Earlier quoted context omitted.

I don't know how to make apps or evaluate code, but with astra I having been making an iOS app on my own for the first time and it's going great. my app is not terribly complex but requires using bluetooth and other intricacies which I thought would be tough. but it's going really well. I'm not asking it to one-shot it though, I'm going feature by feature, testing and building up. yes, at first it would run simulator…

I've been making a macos app with opus 4.8-5 and at first it was great, everything materialized in a week, but when I started tuning stuff and fixing performance problems I have spent a very frustrating month refactoring code where I had to constantly catch llm red-handed and explain and sometimes push obvious ways how to make things work properly (a general knowledge from a completely different stack). In the proces…

were you letting it run for hours like the OP, or doing short tasks and reviewing/testing each one? every once in a while I also ask it to consolidate/summarize docs and stuff like that. we'll see what happens in a week though

Re: Astra for Coding: Why Are We Doing This Again?

#215

Post by trashymctrash »

nnevatie wrote: post #209
taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

> I sincerely don’t understand what the people who say they no longer read any code are doing Welcome to the present. Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.

That’s a huge overstatement. If that were true, then banks would code their core financial services with that approach but they don’t.

Re: Astra for Coding: Why Are We Doing This Again?

#216

Post by gps372 »

TeMPOraL wrote: post #192
gps372 wrote: post #55

Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…

> Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. I think this lesson is getting partially outdated. Yes, you need to be specific about what you want, and with earlier LLMs, you need to had both domain knowledge and some general software dev…

>> No matter how precisely you specify your epic, if the model will find something that contradicts your knowledge/intent, there are good chances it'll get confused and make subtle errors, and you won't realize until much later.

True! hence the need for someone to review the final spec output and own it as their own output. I have also found LLM to be better at debugging and solving 'a' specific problem, which I believe is due to output's surface area to be reviewed is lesser in comparison.

Re: Astra for Coding: Why Are We Doing This Again?

#217

Post by automatic6131 »

codingisfreedom wrote: post #43

I’ve asked Astra to build me an app for a prototype I created quickly using Sonnet. It’s been 2 days and it made no real progress on the actual app. It created docs, scripts, workflows, and it’s doing a bunch of reviewing on every PR. I told it that I just need an MVP. I’m pretty sure an average senior engineer would have finished that task much quicker, and guaranteed with more readable, higher-quality code. Meanwhi…

>Also, I’m really unimpressed by any “one shot” demo that’s out there in the wild. It means nothing for serious software engineering.

If a person, or team of people, can build a demo quickly then it's good odds that they can build the real version (though, famously, not a guarantee). However, it turns out that a machine that can spit out 100 demos of whatever can't actually build the real thing.

Similarly, a chess engine rated to 1000 Elo doesn't play like a 1000 rated human being. The mistakes that each make to reach the equivalent level are different in size, frequency and kind. The thing that makes a human reach a good demo is very close to the skillset to reach the finished article. This isn't so for LLMs but we have yet to update our priors.

Re: Astra for Coding: Why Are We Doing This Again?

#218

Post by bad_username »

taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

Every time I bring this phenomenon up, I hear from HN commenters that it used to be like that, but today it is solved by modern powerful models, harnesses.and workflows. I don't know. At work I still direct and review everything LLMs generate, because if HN are wrong and I am right, I am in huge huge trouble.

Re: Astra for Coding: Why Are We Doing This Again?

#219

Post by neal_jones »

samuell wrote: post #138
gps372 wrote: post #55

Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…

Yeah, I think as a general direction we need to allow a useful collaboration with the agents on the issue board. This is btw why Epiq was developed, to keep the board as code, git-backed, distributed (via an event log mechanism), and with the ability to replay the board, to see what agents actually did: https://ljtn.github.io/epiq

This looks very cool

Re: Astra for Coding: Why Are We Doing This Again?

#220

Post by oblio »

arcanemachiner wrote: post #104
oblio wrote: post #23

Earlier quoted context omitted.

Why even offer the edit tool in that case? Also, what kind of editing could they possible do what wouldn't be possible with POSIX ed?

You can chain a lot more commands together with this technique than with a single Edit tool call.

The funny thing is that... POSIX ed is composable :-)

You can do a gazillion edits with it in one shot.

Of course, LLM edit tools are probably small bits of their custom code, I just find it funny. I wonder if it's a desire for certain technical characteristics that require custom code or just a lack of info on basic tools. Heck, if it's about platform availability, using an LLM to port ed to Windows (for example) should be trivial[1].

* * *

[1] And there are probably a million existing ports. Also, sed, ex, vi, whatever.

Post reply on HN