Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra for Coding: Why Are We Doing This Again?

lucumr.pocoo.org

Post reply on HN
Print view View original

181–190 of 334 posts

Re: Astra for Coding: Why Are We Doing This Again?

#181

Post by OtomotO »

idiliv wrote: post #15
mgrosvenor wrote: post #12

These machines are doing some crazy things to get to the result. That said, I can't help but feel like this is the compilers argument all over again. Are the methods used to get to the result good? No. Is the code that it generates good? No. Does it achieve the goal. Yes. Is it likely to get better with time. Also yes. In my use cases, jobs that would have taken weeks to months are being done in minutes to hours. Inv…

What is the "compilers argument"?

I assume it's the idea that initially you wrote machine code, later assembly and then the "high level languages" started to gain traction.

And for each iteration there were scepticals...

But I am curious myself, what OP meant by this.

Re: Astra for Coding: Why Are We Doing This Again?

#182

Post by recursivecaveat »

eptcyka wrote: post #105
nojs wrote: post #68

This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.” My suspicion is that both OpenAI and Anthropic moved their RL agendas from "being rated as useful according to human feedback" to "succeeds at long hor…

I wouldn't be surprised if they are optimising for producing more code, because in the long term, more existing code means they can sell you more tokens to maintain it.

The incentives are certainly extremely strong. I have read hundreds of AI review comments, and I don't think I've ever seen an unprompted suggestion focused on net reducing code or increasing readability.

Re: Astra for Coding: Why Are We Doing This Again?

#183

Post by whstl »

recursivecaveat wrote: post #154
dakolli wrote: post #137

Earlier quoted context omitted.

Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process. I'm not convinced this style of "agentic engineering" saves much time. I guess if I was oblivious to what good code looks like, and didn't care about maintainability It wouldn't bother me so much, but it legitimately has e…

I'm genuinely not convinced it actually saves time once a full accounting has been made. You get the initial result faster, but then you inflict a super slow and torturous review process on yourself or a teammate. Even if the review manages to bring it up to parity, over time you will keep slowing down as more and more code was never written by the humans directing the agents, so their understanding decays. I at leas…

My team experimented with re-writing from scratch the prototype of complex functionality made by a non-engineering vibe-coder from another team. We didn't look at the code, and barely looked at the result.

It took about 4 days to get a production-ready reviewed code, while it took them 2-3 months to deliver something that another team judged "impossible to review".

The PR for the prototype was closed.

It helps that I'm a domain expert here, as I have a minor degree in the domain, so I can judge better. But the discrepancy is just too high to ignore.

Re: Astra for Coding: Why Are We Doing This Again?

#184

Post by oleggromov »

applfanboysbgon wrote: post #112
nvrmnd wrote: post #75

51 comments so far, the vast majority panning Astra's coding abilities. An uninformed reader may come away with the impression that this isn't an absolutely revolutionary technology that with coding abilities many of us thought were not even going to be possible with language models as recently as a year ago. Yeah, it's not perfect, but it's really good and extrapolating this rate of improvement for 6 months is rathe…

It's impressive technology but not revolutionary. Revolutionary technology would have resulted in, you know, a revolution in software quality. Instead quality keeps going down.

It is revolutionary. Programmers are paid less to work more.

Re: Astra for Coding: Why Are We Doing This Again?

#185

Post by exceptione »

llama-for3ver wrote: post #49
buildbot wrote: post #14

I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…

this is intentional, afaik agents do better with python and alike than the harness tooling.

Using python or any other stone-age approach for search and replace is stupid when your language provides you with a complete, fully typed AST, like .NET does.

Re: Astra for Coding: Why Are We Doing This Again?

#186

Post by FailMore »

arthurlockman wrote: post #84
FailMore wrote: post #83

I think this is a very interesting article because it raises an idea I had not considered: these companies found PMF and huge growth through satisfy the demands of coders, it is interesting if they are in a bind where improving the model in one direction worsens it in others

You’re correct–you actually can’t improve the model in one area without changing the characteristics in every other area. It’s almost like the whole thing is just a lot of linear regression…

As in you change one parameter and the whole equation changes or something else? (I'm bad at math!)

Re: Astra for Coding: Why Are We Doing This Again?

#187

Post by maaaaattttt »

taurath wrote: post #120

When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…

In my experience you can “stop reading the code” if you adhere to prompting the AI aiming for changes that would be reasonable in a PR to review, if you’re using rather opinionated framework(s) to base your work on and explicitly ask for tests (models tend to add them on their own if there are some). I personally still read the output and start by checking if existing tests were modified (kind of a red flag when this unexpectedly happens imo).

When I read posts about AI generating garbage nowadays it’s either because of a small prompt/big ask combination or a lack of an underlying framework.

Re: Astra for Coding: Why Are We Doing This Again?

#188

Post by adamddev1 »

Marazan wrote: post #30

I love this dance we are doing where when people write the "AI models are garbage machines that produce garbage and are no where close to the fantasy being pedalled by the Crypto bros who pivoted to AI" it always has to be caveated with "AI models are useful and I am highly productive with them" It feels like people should just be able to say "This article comes with the standard disclaimer" and just dive into the me…

There's also often the obligatory "well, we still have to use these tools so perhaps we could use them better like this." As if just not using them wasn't an option.

Re: Astra for Coding: Why Are We Doing This Again?

#189

Post by byzantinegene »

nvrmnd wrote: post #75

51 comments so far, the vast majority panning Astra's coding abilities. An uninformed reader may come away with the impression that this isn't an absolutely revolutionary technology that with coding abilities many of us thought were not even going to be possible with language models as recently as a year ago. Yeah, it's not perfect, but it's really good and extrapolating this rate of improvement for 6 months is rathe…

i think the main consensus here is that the actual performance is not indicative of the benchmark performance (which supposedly outperforms the previous iterations)

Re: Astra for Coding: Why Are We Doing This Again?

#190

Post by klibertp »

What's funny is that with Sol, I added an instruction to AGENTS.md in one project to prefer sed/python ("deterministic tools" in general) for moving code instead of deleting it and rewriting it elsewhere from memory, because otherwise it butchered comments. After switching to Astra, I saw it suddenly do this for all edits in all projects, which isn't great: the second argument to `replace` is still written "from memory", but now you need to unravel the Python script before you can understand what was actually changed.
Post reply on HN