I think experiments like these are exciting, but they only work when you have the full suite against which you calibrate.
What I'd love to see is a small group of humans steering a million agents.
Agent swarms and the new model economics
131–140 of 159 posts
Re: Agent swarms and the new model economics
#132Earlier quoted context omitted.
It's hard not read such quotes and immediately think of the Infinite Monkey Theorem. https://en.wikipedia.org/wiki/Infinite_monkey_theorem
Isn't that the core fundamental of AI? Constrain the results to a latent space which has all the probable answers and then pick up whichever token makes sense based on your random sampling? I feel like the whole concept of harness and LLMs IS to pick random tokens which make sense. Think about this as a solution space (a very complex one), then the harness and the LLM are moving towards the local minima at which the…
Yes, agentic coding is machine learning. Not because of the underlying language models, but because it is gradient descent, where the test harness provides a gradient for the agent to traverse.
Re: Agent swarms and the new model economics
#133Earlier quoted context omitted.
Wait, why isn't context cached with /loop?
Because it's typically cached for 5min (1hr is a setting you have to explicitly opt into), and very few people run loops at a cadence of < 5 mins.
https://code.claude.com/docs/en/prompt-caching#on-a-claude-s...
"On a Claude subscription, Claude Code requests the one-hour TTL automatically. Usage is included in your plan rather than billed per token, so the longer TTL costs you nothing extra and only affects how long your cache stays warm. If you’ve gone over your plan’s usage limit and Claude Code is drawing on usage credits, you are billed for that usage, so Claude Code automatically drops the TTL to five minutes."
Re: Agent swarms and the new model economics
#134Am I reading this right? Opus + Composer did a comparable job to Fable, at ~1/19th the price, and half the LoC?
Yea, looks like it (at least for their v2). Fable did much better in V1, but once they added their tooling around it, Opus + Composer ended up doing better (>0.8 Grade on the SQLite chart) for the final product. Granted, Fable reached a 'good' result (0.7~) in a much faster timeframe.
Re: Agent swarms and the new model economics
#135Earlier quoted context omitted.
It's hard not read such quotes and immediately think of the Infinite Monkey Theorem. https://en.wikipedia.org/wiki/Infinite_monkey_theorem
If at the end Shakespeare-level literature is produced, does it matter whether we arrived there by random keystrokes?
Yes it will be a terrible waste of time since dedicated humans can do it faster and produce better stuff. I would prefer literature created like that instead of just random monkeys writing what they don't understand. We also lose the ability for writers and creators to get better at their craft and produce unique works instead of just randoming spamming keys for millions of years
Re: Agent swarms and the new model economics
#136This is almost a year behind Steve Yegge's first post on beads. Gas Town and Gas City provide orchestration for the swarm. So far I haven't seen a perfect implementation but this idea isn't new.
Everything Steve yegge has done has been trash. That's why nobody is talking about beads or gas town. It was clear even in the beginning that it was a borderline AI-psychosis-fueled trash fire.
Agreed ... I'm not talking about the implementation but rather the idea that this research is novel.
Re: Agent swarms and the new model economics
#137Am I reading this right? Opus + Composer did a comparable job to Fable, at ~1/19th the price, and half the LoC?
Point is, frontier models don't necessarily are better than a swarm of non-frontier ones for these scoped problems. IMO frotnier models excel at underspec'd or more complex problems where ideation and exploration are key (and that's where they become crazy expensive).
Re: Agent swarms and the new model economics
#138If we take coding off the table i don't see the case for large agent swarms. More is not always better in my experience experimenting with agents
If you can't see cases for it you may need some glasses. Replace the graph in the docs of Top-level planner -> mid-level planners -> worker leaf nodes and you get an org chart of CEO -> executives, N-levels of management, and ICs at the leafs. It directly maps to almost any office-based company that can operate completely digitally.
It's obviously in its infancy but fully agentic companies are becoming a possibility, given the right harness, models and tools
Re: Agent swarms and the new model economics
#139Amazing test. I wish I could spend $2000 of compute on having my own swarm build some weird piece of software with 1000 commits per second.
Perhaps you can get hired by a company to do that. You can't compare yourself as an individual with a company with a large budget.
Re: Agent swarms and the new model economics
#140Earlier quoted context omitted.
Honestly judging from the volume of his output, a ton . It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.
The director James Cameron stands out because he has only produced huge box office hits. Extremely hard to do like you said, and most artists and auteurs would be happy to have one great work they can be remembered for, even if it's amidst a dozen forgettable pieces.
Sometimes the forgettable pieces come later. In JC's case, that's Avatar 2