A case study in testing with 100+ Claude agents in parallel

But our reaction to it has been to say "ok, well the best practice in software engineering is to make small, well-isolated components anyway, so what if we did that?"

We've been trying to really break things apart into smaller pieces (and that's even evident in mngr, where much of the code is split out into separate plugins), and have been having a ton of success with it.

I realize that that might not be an option for more brownfield / existing / legacy projects, but when making something new, I've really been enjoying this way of building things.

And it is great! Really! Reading your post I was thinking if I could not do the same to write tests in an automated way in project I am working on. It would be awesome!

Though in an other hand we are living in a corporate, capitalistic, and a lot inhumane economic system. If this way of automation would work and deliver consistent output in a way of working software for 2 or 3 years, how long it would take to C-level suits to figure out that it is way better to have 2 or 3 Product Owners and maybe one Designer to write description of the entire programme and then just feed it to one of those automation pipeline? If tech giants will price product like that reasonably and it will work actually, how long it will be till it will cause entire industry to collapse and you will be able to produce software by paying to those tech giants? And it there will be like 5 of those only in the entire world - because nobody else will have enough GPUs. How soon till they came to agreement and split the world in areas of monopoly:

- if your company is in Asia you can either buy your application from Google or Alibaba.

In a world when everything is done in a computer via the software, such concentration of power would be bad for everyone.

Of course I doubt it will come that, simply because this would be very hard to achieve with our level of technology and some human involment will be necessary. But maybe I am kiding myself and I will loose my job entirely in few years along with tens of thousands other Software Engineers in a few years.

How will we know what good software looks like if we no longer write assembler?

> Imagine looking at someone driving a car for 20 years. Will it be enough for you to drive a car yourself?

You don't have to drive stick to be able to drive.

Whatever the economically important functions are, the miracle of capitalism will find a way to staff it and solve it.

People fill all the gaps. No problem goes uninvestigated, no opportunity goes ignored.

At the end of the day we're delivering value. We'll be judged on value creation, and that'll map itself to whatever the tools of the day happen to be.

When they don't merge cleanly, it is time for human intervention, and the integration step would leave traces on which branches failed to merge.

Finally, when you do need to debug individual agents:

- Because mngr is, at the low level, just managed tmux sessions (local and remote), it's very easy to just attach to those sessions (`mngr connect`). It works even if the agent has been stopped, because mngr remembers enough about an agent to resurrect it.

- `mngr message` also allows you batch-message a bunch of agents. So if you do need to resume a lot of agents, you can experiment on one agent, figure out a good prompt, and then batch-message every other agent.

In this testing scenario, most agents don't actually require human intervention, and we've found that just connecting to a few individual agents to resolve problems is smooth and easy enough.