I can't point my finger to anything right now, but I feel that the quality of code still matters because afterall LLMs are trained on what we did, so it feels natural to me to still have them write the code in a good manner (DDD, SOLID, etc.) especially the names and imports, those are very heavy in the context, to help them out
Astra for Coding: Why Are We Doing This Again?
131–140 of 334 posts
Re: Astra for Coding: Why Are We Doing This Again?
#132Ironically I burned out Fable usage early this week because of Astra using it to run inane full codebase reviews over one line changes, so I have been using Astra extensively. We need a word for “potentially highly capable, but in reality an idiot savant” to describe certain models. No, I don’t need you to write a tmux emulator in bash to test your changes bro, just ask me to run the command.
Exactly this. "I need tool objdump but pacman gcc failed because of no sudo password. Let me write compiler, binutils and disassembling framework"
The next model will probably run simulation of a Universe to get a command output
Re: Astra for Coding: Why Are We Doing This Again?
#133Earlier quoted context omitted.
What is the "compilers argument"?
Back in the day, the argument was that compilers produce unreadable assembly, so people used to writing assembly were arguing against the use of compilers. Compilers also had bugs, so we still had to debug the assembly to understand how to fix the problem. Nowadays, almost nobody has to resort to those steps, except of course compiler developers. But that is just a testament to the quality of compilers. Comparing LLM…
And in general, especially with something like C, the correspondence between the compiled code and source code isn’t very “lossy”. You don’t get one-to-one correspondence but it’s not too bad and you’re not introducing massive uncertainty by running it through a compiler. With LLMs though all bets are off… you’re not gonna know what you end up with
Re: Astra for Coding: Why Are We Doing This Again?
#134Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…
> Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. Around February you could get away with very vague prompts to Claude. I feel like models have regressed since
Probably because you were impressed by whatever it produced to fill in the gaps at the time. But if you expect the models to read your mind you're going to have a bad time.
Re: Astra for Coding: Why Are We Doing This Again?
#135> And potentially as a byproduct of enabling all of this, you can now slop your way to a one-shot 3D game over the weekend which looks impressive. I think we've finally reached a weird point where AI has effectively reduced the amount of competition that real game developers have to endure. Nothing unravels faster than a game project being built with AI. You can achieve impressive results in a day, but you can't get…
> LLMs will never be able That is a very bold claim, unless you meant "current LLMs".
These models aren't really LLMs -- they don't just operate on text tokens. They often include vision models and in some cases audio models. That means that they can better associate the meaning of images and words together so that when someone says "make this button blue" or "create a 3D model of a rocket" they have some level of understanding of what that is and what needs to be done.
The key question is how good that understanding is. For example, a model would likely have a good understanding of various named colours and hex values (e.g. from the HTML specs, X11 specs, and various colour comparison websites) such that it could reasonably correlate that to a CSS entry. It's not clear if/how well a model would identify that given an image, though it should be easy to generate a dataset of image to colour name and/or hex code for training and evaluation.
What's more interesting is whether these frontier models are at their core transformer models, whether they use residual streams to facilitate learning, and whether they are using some other as yet unpublished architecture that gives them an edge.
Re: Astra for Coding: Why Are We Doing This Again?
#136内卷/involution seems like one possible kind of "recursive self-improvement".
A circle is also an exponential: y = i^x.
(I have no idea what will happen. 内卷 or intelligence explosion both seem plausible.)
Re: Astra for Coding: Why Are We Doing This Again?
#137When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process.
I'm not convinced this style of "agentic engineering" saves much time. I guess if I was oblivious to what good code looks like, and didn't care about maintainability It wouldn't bother me so much, but it legitimately has effects my "mental health".
Re: Astra for Coding: Why Are We Doing This Again?
#138Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…
Yeah, I think as a general direction we need to allow a useful collaboration with the agents on the issue board.
This is btw why Epiq was developed, to keep the board as code, git-backed, distributed (via an event log mechanism), and with the ability to replay the board, to see what agents actually did:
Re: Astra for Coding: Why Are We Doing This Again?
#139> I wonder if there is really enough signal going to the training processes for “a human understands what is going on”
Unverifiable, un-scalable, no.
Re: Astra for Coding: Why Are We Doing This Again?
#140There is something odd, I've got single astra session that's now running for... 4d 13h 10m and still going.
How does that translate to cost? I am unfamiliar with OpenAI pricing models.