[flagged]
...at least until we get real Test-Time Training (TTT) that encodes the state into model weights. If vast amounts of human knowledge can be compressed into ~400GB for frontier models, it's easy to imagine the same for our entire context
Levels of Agentic Engineering
131–140 of 152 posts
Re: Levels of Agentic Engineering
#132> Level 8: Autonomous Agent Teams Nobody has mastered this level yet, though a few are pushing into it. It's the active frontier
Speak for yourself.
Also Level 7 is a misunderstanding of why plan mode is actually used even though one-shot works perfectly
Re: Levels of Agentic Engineering
#133Re: Levels of Agentic Engineering
#134Earlier quoted context omitted.
Your points are all valid, but it doesn’t really change the situation that was being discussed: an AI company trying to enter completely new markets just because they can write software for it is hardly some sort of automatic win. They’re much more likely to fail than succeed. I mentioned sales and marketing but there’s a whole lot more as well. Basically, it involves creating an entire subsidiary. Perhaps the time w…
It does raise the question of where in the future will companies compete. What's the balance going to be between, 'connecting customers to product' and 'making differentiated product'? In theory, if customers have perfect information ( ignoring a very large part of sales is emotional ), then the former part will disappear. However the rise of the internet, and perhaps AI agents shopping on your behalf, hasn't really…
There's also a world where "all companies have access to the software factory so sales and entrepreneurship in software disappears entirely."
But in that scenario it's hard to see where the unwinding stops. What are these other companies doing and which parts of it actually need humans if the "agents" are that good? Marketing? No. Talking to customers? No. Support? No. Financial planning and admin? No. Manufacturing? Some, for now. Shipping physical goods? For now. What else...
At some point where even are your customers?
Re: Levels of Agentic Engineering
#135Earlier quoted context omitted.
How much money have you made with this approach
I think the opposite question is more prevalent, how much money have you spent?
Not a small amount :)
I spend $140/mo on Anthropic + OpenAI subs and I use all my tokens all the time.
I've started spending about $100/week on API credits, but I'd like to increase that.
Re: Levels of Agentic Engineering
#136Earlier quoted context omitted.
I got my own level 8 factory working in the last few days and it’s been exhilarating. Mine is based on OpenAI’s Symphony[1], ported to TypeScript. Would be happy to swap war stories. @gmail.com
How much money have you made with this approach
none yet!
Re: Levels of Agentic Engineering
#137[flagged]
strong agree. I always have the LLM put an actual markdown doc in a docs/plans/ folder before starting work. I often, but not always review it.
Aside: it also helps for code review! Review bots can point out the diff between plan and implementation.
Some examples for the curious: https://github.com/sociotechnica-org/symphony-ts/tree/main/d...
Re: Levels of Agentic Engineering
#138[flagged]
strong agree. I always have the LLM put an actual markdown doc in a docs/plans/ folder before starting work. I often, but not always review it. Aside: it also helps for code review! Review bots can point out the diff between plan and implementation. Some examples for the curious: https://github.com/sociotechnica-org/symphony-ts/tree/main/d...
[flagged]
Re: Levels of Agentic Engineering
#139I'm practicing around 5/6, I've found that adding "skills" and mcp can sometimes negatively impact the process as much as help, so I've been somewhat constrained on overdoing it. As for 6, mostly just setup guardrails on repeat testing instructions and how to run/test/retest certain things... also to not just update tests when broken, but confirm the designed behavior.
Moving past that, I'm not sure that I really trust it... I feel that manual review of product behavior and code matters a lot. AI agents often make similar mistakes to real people in leaking abstractions or subtle mistakes with security... So I do review almost everything, at least at the level where a feature PR makes sense. Though an AI pass at that can help too.
Re: Levels of Agentic Engineering
#140What level is copy pasting snippets into the chatgpt window? Grug brained level 0? I sort of prefer it that way (using it as an amped up stackoverflow) since it forces me to decompose things in terms of natural boundaries (manual context management as it were) and allows me to think in terms of "what properties do I need this function to have" rather than just letting copilot take the wheel and glob the entire projec…
1.8: chat ide the slow way :) This is also where I do most of my AI use. It’s the safe spot where I’m not going to accidentally send proprietary info to an unknown number of eyeballs(computer or human). It’s also just cumbersome enough that I’m not relying on it too much and stunting my personal ability growth. But I’m way more novice than most on here.
I've found it's easy enough to have AI scaffold a working demo environment around a single component/class that I'm actually working on, then I can copy the working class/component into my "real" application. I'm in a pretty locked down environment, so using a separate computer and letting the AI scaffold everything around what I'm working on is pretty damned nice, since I cannot use it in the environment or on the project itself.
For personal projects, I'm able to use it a bit more directly, but would say I'm using it around 5/6 level as defined here... I've leaned on it a bit for planning stages, which helps a lot... not sure I trust swarms of automated agents, though it's pretty much the only way you're going to use the $200 level on Claude effectively... I've hit the limits on the $100 only twice in the past month, I downgraded after my first month. And even then, it just forced me to take a break for an hour.