Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agent swarms and the new model economics

cursor.com

Post reply on HN
Print view View original

121–130 of 159 posts

Re: Agent swarms and the new model economics

#121

Post by notahan »

jgalt212 wrote: post #11
htrp wrote: post #2

>The browser swarm from earlier this year peaked at roughly 1,000 commits per hour on Git. The new system peaks at around 1,000 commits per second. >To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coor…

It's hard not read such quotes and immediately think of the Infinite Monkey Theorem. https://en.wikipedia.org/wiki/Infinite_monkey_theorem

Isn't that the core fundamental of AI? Constrain the results to a latent space which has all the probable answers and then pick up whichever token makes sense based on your random sampling?

I feel like the whole concept of harness and LLMs IS to pick random tokens which make sense. Think about this as a solution space (a very complex one), then the harness and the LLM are moving towards the local minima at which the code is complete. This is akin to how we think about AI anyways.

Re: Agent swarms and the new model economics

#122

Post by bscphil »

NitpickLawyer wrote: post #97
mrtesthah wrote: post #84

Earlier quoted context omitted.

To test this hypothesis you would need to look at the initial version before subsequent agents started refactoring it.

Why tho? The whole point of agentic is that it takes several passes to get to a result. If step a) is a non-idiomatic transpile, but step b) c) and ... z) make it idiomatic and performant and efficient, why look at step a)? That was just the beginning. The idea with "agents" and "swarms" and dynamic this and that is that you only look at the last step. This strictly follows the early agentic usage, where people were…

Because the point of the test is to see what the agentic swarm approach is capable of. If your test assumes that a version of the thing you are trying to build is available in the training data, or can't determine if it matters whether the thing you're building is in the training data, then it's not a very useful test. Most of the time when you consider agentic approaches, you're not trying to rebuild something the agents involved are trained on.

Re: Agent swarms and the new model economics

#123

Post by gedy »

whatever1 wrote: post #30
jgalt212 wrote: post #11

Earlier quoted context omitted.

It's hard not read such quotes and immediately think of the Infinite Monkey Theorem. https://en.wikipedia.org/wiki/Infinite_monkey_theorem

If at the end Shakespeare-level literature is produced, does it matter whether we arrived there by random keystrokes?

Yes, because the folks waiting slack-jawed for the monkeys to finish will no longer have the skills or discernment to know what Shakespeare looks like.

Re: Agent swarms and the new model economics

#124

Post by notahan »

These tests are interesting but I can't help feeling that these are benchmaxxing results. "We made our harness make sqlite in rust" sounds like a much easier probelm statement than having to recreate an application like facebook. I think the concept of having to integrate is the real challenge. Re-writing code in new languages based on decent docs is a difficult but less useful measure of how AI can help replace engineers.

Imagine this: you need to build a new CRUD app but your AI agent spends 20M toks on just writing it's own custom login function, that's a waste of time and money. I would not like to have a new system developed for me in such cases. I think we need a test where the agents are more specifically tasked to find technologies worth integrating and hence write the fewest lines of code.

Re: Agent swarms and the new model economics

#125

Post by patmorgan23 »

Schlagbohrer wrote: post #109
mock-possum wrote: post #82

Earlier quoted context omitted.

Honestly judging from the volume of his output, a ton . It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.

The director James Cameron stands out because he has only produced huge box office hits. Extremely hard to do like you said, and most artists and auteurs would be happy to have one great work they can be remembered for, even if it's amidst a dozen forgettable pieces.

He's only published blockbuster hits. But I'm sure he has lots of practice films that have never seen the light of day?

Re: Agent swarms and the new model economics

#126

Post by jackb4040 »

wrs wrote: post #45
kgeist wrote: post #31

Earlier quoted context omitted.

Even if no Rust code for it was seen during training, an LLM can trivially transpile SQLite's C codebase to Rust on the fly. For example, I just asked ChatGPT to write John Carmack's famous Fast Inverse Square Root algorithm in Erlang, without searching online or thinking, and it transpiled it immediately (while also extracting the knowledge in the same step). SQLite's semantics/code are stored in the middle layers o…

Then you would expect the implementation to be structured the same as SQLite, and having glanced at the result, it looks like at least some things aren't. For example, it seems to use an operator-tree executor rather than SQLite's bytecode interpreter.

The point is that the model has the source code in its weights. Therefore it's misleading to focus on being "only given the documentation", when it could produce a SQLite implementation in rust with no documentation at all. The wording is designed to create a marketing impression that it can do spec-driven development for a novel project, but this example doesn't actually support that.

It's like a diet system advertising how much weight a spokesperson lost on their system, without disclosing that they also went on Ozempic at the same time.

Re: Agent swarms and the new model economics

#127

Post by lumost »

notahan wrote: post #124

These tests are interesting but I can't help feeling that these are benchmaxxing results. "We made our harness make sqlite in rust" sounds like a much easier probelm statement than having to recreate an application like facebook. I think the concept of having to integrate is the real challenge. Re-writing code in new languages based on decent docs is a difficult but less useful measure of how AI can help replace engi…

The nature of ML is to reach for easily measured proxy problems which can then be optimized. Why worry about a backoffice integrated micro-service you have no data on when you aren't sure that you can design and implement a rewrite of an OSS project?

Re: Agent swarms and the new model economics

#128

Post by hexaga »

mock-possum wrote: post #82
timacles wrote: post #69

Earlier quoted context omitted.

How much slop you figure Shakespeare produced?

Honestly judging from the volume of his output, a ton . It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.

And then there are those like J.S. Bach.

Re: Agent swarms and the new model economics

#129

Post by NitpickLawyer »

bscphil wrote: post #122
NitpickLawyer wrote: post #97

Earlier quoted context omitted.

Why tho? The whole point of agentic is that it takes several passes to get to a result. If step a) is a non-idiomatic transpile, but step b) c) and ... z) make it idiomatic and performant and efficient, why look at step a)? That was just the beginning. The idea with "agents" and "swarms" and dynamic this and that is that you only look at the last step. This strictly follows the early agentic usage, where people were…

Because the point of the test is to see what the agentic swarm approach is capable of. If your test assumes that a version of the thing you are trying to build is available in the training data, or can't determine if it matters whether the thing you're building is in the training data, then it's not a very useful test. Most of the time when you consider agentic approaches, you're not trying to rebuild something the a…

I see, that makes sense. Thank you.
Post reply on HN