Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra for Coding: Why Are We Doing This Again?

lucumr.pocoo.org

Post reply on HN
Print view View original

81–90 of 334 posts

Re: Astra for Coding: Why Are We Doing This Again?

#81

Post by d5lt5 »

weird-eye-issue wrote: post #42
d5lt5 wrote: post #26

Earlier quoted context omitted.

Sounds like you don't have enough experience with coding agents. Deterministic scripts must always be preferred instead of LLM tool calls. In fact, you should instruct your agents to write code to execute instead of letting them call tools.

Sounds like you completely lack all reading comprehension ability LLMs sometimes like to execute one-off Python scripts to make edits to files rather than just calling the edit tool directly. Both are tool calls so saying that you should have it write code instead of doing tool calls makes no sense because writing code is a tool call for it...

[flagged]

Re: Astra for Coding: Why Are We Doing This Again?

#82

Post by quikoa »

prerok wrote: post #46
idiliv wrote: post #15

Earlier quoted context omitted.

What is the "compilers argument"?

Back in the day, the argument was that compilers produce unreadable assembly, so people used to writing assembly were arguing against the use of compilers. Compilers also had bugs, so we still had to debug the assembly to understand how to fix the problem. Nowadays, almost nobody has to resort to those steps, except of course compiler developers. But that is just a testament to the quality of compilers. Comparing LLM…

There is also a predictable relation between the input and output of a compiler w.r.t. the semantics of a programming language. Natural languages are ambiguous leaving room for the implementation to diverge that may not be obvious at first glance.

Re: Astra for Coding: Why Are We Doing This Again?

#83

Post by FailMore »

I think this is a very interesting article because it raises an idea I had not considered: these companies found PMF and huge growth through satisfy the demands of coders, it is interesting if they are in a bind where improving the model in one direction worsens it in others

Re: Astra for Coding: Why Are We Doing This Again?

#84

Post by arthurlockman »

FailMore wrote: post #83

I think this is a very interesting article because it raises an idea I had not considered: these companies found PMF and huge growth through satisfy the demands of coders, it is interesting if they are in a bind where improving the model in one direction worsens it in others

You’re correct–you actually can’t improve the model in one area without changing the characteristics in every other area. It’s almost like the whole thing is just a lot of linear regression…

Re: Astra for Coding: Why Are We Doing This Again?

#85

Post by [deleted] »

exitb wrote: post #20

My own observations are that I used to target turn lengths of 10-15 minutes and these new models (since 5.6) extended that a bit to ~25 minutes, as they tend to do more tests and reviews. Targeting hours-long turns makes as much sense, as putting on cruise control and going to sleep.

[deleted]

Re: Astra for Coding: Why Are We Doing This Again?

#86

Post by klipt »

nojs wrote: post #68

This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.” My suspicion is that both OpenAI and Anthropic moved their RL agendas from "being rated as useful according to human feedback" to "succeeds at long hor…

So the AI equivalent of the socially stunted but brilliant researcher?

Re: Astra for Coding: Why Are We Doing This Again?

#87

Post by SCUSKU »

I finally ran Astra on a dashboard feature today, and while I was vibing it looked great, but then when it came time to actually read the code I was appalled because it was the worst looking code I had seen from an LLM since like last November. I mean it was just the definition of slop, not re-using anything, super terse with mega-ternaries, re-writing functions that should be using standard library packages, etc, etc. I think for coding I'm gunna stick with the 5.6 series of models, or maybe try out Anthropic again...

Re: Astra for Coding: Why Are We Doing This Again?

#88

Post by sbt »

nvrmnd wrote: post #75

51 comments so far, the vast majority panning Astra's coding abilities. An uninformed reader may come away with the impression that this isn't an absolutely revolutionary technology that with coding abilities many of us thought were not even going to be possible with language models as recently as a year ago. Yeah, it's not perfect, but it's really good and extrapolating this rate of improvement for 6 months is rathe…

There was a big jump around new year, but they seem to have flatlined since them. Just my experience.

Re: Astra for Coding: Why Are We Doing This Again?

#89

Post by [deleted] »

troupo wrote: post #66
gps372 wrote: post #55

Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…

> Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. Around February you could get away with very vague prompts to Claude. I feel like models have regressed since

[deleted]

Re: Astra for Coding: Why Are We Doing This Again?

#90

Post by drybjed »

nojs wrote: post #68

This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.” My suspicion is that both OpenAI and Anthropic moved their RL agendas from "being rated as useful according to human feedback" to "succeeds at long hor…

I wonder if we will start using LLMs to translate the output of other LLMs to make it more palatable for humans.
Post reply on HN