51–60 of 75 posts
Earlier quoted context omitted.
What most people leave out of the conversation is the element of time. We are early. The technology continues to improve. Is it perfect now? Absolutely not. Will it continue to get better at a surprising pace? Absolutely yes. So, just adjust your framing from "we're there" to "we're getting there" and all of your concerns go away.
Read "no silver bullet" article.
Earlier quoted context omitted.
Read "no silver bullet" article.
I like your name, which article is this?
In such systems, producing code is the easy part, verification is hard. Verification via test cases just feels like the same agent validating its own assumptions. Wondering what the actual verification loop looks like once you start taking these systems to production.
In such systems, producing code is the easy part, verification is hard. Verification via test cases just feels like the same agent validating its own assumptions. Wondering what the actual verification loop looks like once you start taking these systems to production.
Agreed, we had agents writing standard integration/unit/e2e tests but turned out to be a ton of work reviewing just that and hard to wrap one’s head around as a developer. We ended up building our own test platform to reduce the human bottleneck in the verification loop. Takes the form of a hermetic test environment that run all our code and infra in a way that matches production. Our tests interact with it from the…
People who build software factory are in for a disappointing future. Spoken as someone who uses AI day to day for job and personal projects, I've came across a few super gnarly bugs already, ranging from frontend React apps (yes, believe it or not, frontend is far from solved) to embedded that Claude can't solve. Those bugs are feature breaker. My primary skillset is fullstack/frontend leaning, not embedded. Granted,…
However, if you just think of these kinds of LLM driven software factories as having a limit in terms of complexity (ie. some asymtotic / logarithmic increase in cost beyond aome complexity threshold), it still is quite a useful system to have (despite being admittably more niche) because it lets you quickly explore various product ideas, user experiences, and featuresets.
My personal take is that the best approach is to first prototype products as vibe coded slop using a software factory like this to understand the user experience, featureset, and product concepts. Then, once you have a clearer understanding you can properly design out the system and more carefully rebuild it either from scratch with a more costly model and clear architectural details or strip out the core systems and redesign them in large refactors.
Earlier quoted context omitted.
I just wrote a Twitter thread about how LLMs are like chainsaws [0] In short: - they are very powerful - beginners feel like they are just as good as experts - but they can cause a LOT of damage if you don't know what you are doing - experts will get way more done and beginners may cut off a limb or drop a tree on their car 0 - https://x.com/alexpotato/status/2090872063464509912?s=20
People have been making analogies like this since 2023, hardly worth a link to a random twitter thread.
In such systems, producing code is the easy part, verification is hard. Verification via test cases just feels like the same agent validating its own assumptions. Wondering what the actual verification loop looks like once you start taking these systems to production.
Agreed, we had agents writing standard integration/unit/e2e tests but turned out to be a ton of work reviewing just that and hard to wrap one’s head around as a developer. We ended up building our own test platform to reduce the human bottleneck in the verification loop. Takes the form of a hermetic test environment that run all our code and infra in a way that matches production. Our tests interact with it from the…
Earlier quoted context omitted.
Agreed, we had agents writing standard integration/unit/e2e tests but turned out to be a ton of work reviewing just that and hard to wrap one’s head around as a developer. We ended up building our own test platform to reduce the human bottleneck in the verification loop. Takes the form of a hermetic test environment that run all our code and infra in a way that matches production. Our tests interact with it from the…
You could let a agent drive the browser verification