201–210 of 334 posts
Earlier quoted context omitted.
What you say is true, the comparison indeed doesn't hold. But is it relevant? does it matter from a product perspective if LLMs are non-deterministic. You don't need to one shot the correct result, english is ambiguous and LLMs non-deterministic, but you can iterate. If it's possible to iterate fast and cheap enough, even ambiguous language can produce the results you want, given enough iterations. There are a lot of…
I think it matters, because nowadays we don't look at assembly any more. I mean, I don't recall the last time I was tracking down a compiler bug but it's definitely been more than 15 years ago. We do have to look at the LLMs' output, though, and, as you already pointed out, iterate to get the correct results. What this means is that the output must still be readable, must be analyzed by someone and I don't see it goi…
But from a broad market and product perspective, for most things you don't need to look at the code. If the product kinda does what it's supposed to.
For example, in my game projects I don't look at the CMakeLists anymore, or python scripts that move assets here and there, I can run my game and just see that it did what I expect it to do (renders assets etc).
Similar with frontend, I don't care that much what the code looks like anymore, mostly that the site looks and feels as I expect it, and the correct network calls are happening.
TLDR; I'm thinking there are levels to this, in some projects it matters, in others it doesn't, it's kinda two different things. Programming wasn't replaced, LLMs just brought a new paradigm of doing things on the side.
I'm just rambling at this point, my thoughts on this are not super clear, sorry for that :D
> My software factory was intentionally set up to let the model decide the how of the workflow entirely. It was free to manage its own context and could maintain its own records in an agent-notes folder.
The experiment becomes a crapshoot. What are we evaluating? The ability of the thing to create it's own factory workflow? Or adding virtual threads to Python?
Astra is clearly both formidable and imperfect. Anyone who understands how to get the best out of it will have a strong advantage.
(For me - my CC is stuck in Sonnet and consumes Trello cards that have passed readiness criteria)
Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can…
> in fact more specific than usual. At which point you might as well write the code yourself and get a deterministic result faster, better and cheaper.
Where you would have a point is if you'd say we have excellent tooling for wrangling code, but less tooling and tradition to write and manage specs.
The hype machine is this technology's worst enemy. When I zoom out and look at things objectively, it's kind of crazy what we have at our fingertips, we can talk to our computers in plain and even vague human languages and have the computers actually accomplish what we ask of it! It's literally sci-fi magic come to life, and the nerd in me finds it the coolest thing ever. But then the industry and the companies invol…
The more it evolves, the clearer it gets that the humankind is deeply in love with its own death.
Earlier quoted context omitted.
What is the "compilers argument"?
Lots of really silly people love to compare LLMs to compilers. "You don't look at the compiled code either" and "Back in the day, people also had negative reactions to compilers and wanted to keep writing assembly by hand" and other such nonsense.
The slop machine reads lousy natural human language that can mean different things in different contexts. That lousy language is then statistically probed for the most likely output correspondence, producing shit that needs to be externally verified.
When the code is shitty it becomes harder and harder for the models to make changes and this grinds progress down to a halt - this has been my experience with “factories” trying them and doing refining steps every few months. I sincerely don’t understand what the people who say they no longer read any code are doing, because it must be somewhat trivial to not run headlong into these issues that stack up time after ti…
Welcome to the present.
Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.
There is something odd, I've got single astra session that's now running for... 4d 13h 10m and still going.
What are you having it do?