Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Sid: Many-agent simulations toward AI civilization

github.com

Post reply on HN
Print view View original

71–80 of 173 posts

Re: Project Sid: Many-agent simulations toward AI civilization

#71

Post by Reubend »

caetris2 wrote: post #38

I've reviewed the paper and I'm confident this paper was fabricated over a collection of false claims. The claims made are not genuine and should not be taken at face value without peer review. The provided charts and graphics are sophisticated forgeries in many cases when reviewing and vetting their applicability to the claims made. It is currently not possible for any kind of LLM to do what is being proposed, while…

Yeah, I haven't looked into this much so far but I am extremely skeptical of the claims being made here. For one agent to become a tax collector and another to challenge the tax regime without such behavior being hard coded would be extremely impressive.

Re: Project Sid: Many-agent simulations toward AI civilization

#72

Post by afro88 »

caetris2 wrote: post #52
afro88 wrote: post #43

Earlier quoted context omitted.

> this paper seems indicate that election-related activities were coordinated by groups of AI agents in a simulation I mean, that's surely within the training data of LLMs? The effectiveness etc of the election activities is likely very low. But I don't think it's outside the realms of possibility that the agents prompted each other into the latent spaces of the LLM to do with elections.

LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here. Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false. The ideas here are not supported by any…

Perhaps I've made a big assumption / oversimplification about how this works. But..

> LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here

Yes. I never said they were stateful? The context given is the state. And training data is hugely important. Once upon a time there was a guy that claimed ChatGPT could simulate a command line shell. "Simulate" ended up being the wrong word. "Largely hallucinate" was a more accurate description. Shell commands and sessions were for sure part of the training data for ChatGPT, and that's how it could be prompted into largely hallucinating one. Same deal here with "election activities" I think.

> Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false.

Well no, they can always trim the data put into the context. And then the agents would start "forgetting" things and the "election activities" would be pretty badly "simulated".

Honestly, I think you're right that the paper is misleading people into thinking the system is doing way more than it actually is. But you make it sound like the whole thing is made up and impossible. The reality is somewhere in the middle. Yes they set up hundreds of agents, they give the agents data about the world, some memory of their interactions, and some system prompt to say what actions they can perform. This led to some interesting and surprising behaviours. No, this isn't intelligence, and isn't much more than a fancy representation of what is in the model weights.

Re: Project Sid: Many-agent simulations toward AI civilization

#73

Post by afro88 »

Reubend wrote: post #71
caetris2 wrote: post #38

I've reviewed the paper and I'm confident this paper was fabricated over a collection of false claims. The claims made are not genuine and should not be taken at face value without peer review. The provided charts and graphics are sophisticated forgeries in many cases when reviewing and vetting their applicability to the claims made. It is currently not possible for any kind of LLM to do what is being proposed, while…

Yeah, I haven't looked into this much so far but I am extremely skeptical of the claims being made here. For one agent to become a tax collector and another to challenge the tax regime without such behavior being hard coded would be extremely impressive.

You can imagine a conversation with an LLM getting to that territory pretty quickly if you pretend to be an unfair tax collector. It sounds impressive on the surface, but in the end it's all LLMs talking to each other, and they'll enit whatever completions are likely given the context.

Re: Project Sid: Many-agent simulations toward AI civilization

#75

Post by Philpax »

caetris2 wrote: post #52
afro88 wrote: post #43

Earlier quoted context omitted.

> this paper seems indicate that election-related activities were coordinated by groups of AI agents in a simulation I mean, that's surely within the training data of LLMs? The effectiveness etc of the election activities is likely very low. But I don't think it's outside the realms of possibility that the agents prompted each other into the latent spaces of the LLM to do with elections.

LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here. Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false. The ideas here are not supported by any…

> LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here.

That's not what they said. They said that a LLM knows what elections are, which suggests they could have the requisite knowledge to act one out.

> Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false.

No, it doesn't. They aren't passing in all prior context at once: they are providing relevant subsets of memory as context. This is a common technique for language agents.

> Agentic systems are not well-suited to achieve any of the things that are proposed in the paper, and Generative AI does not enable these kinds of advancements.

This is not new ground. Much of the base social behaviour here comes from Generative Agents [0], which they cite. Much of the Minecraft related behaviour is inspired by Voyager [1], which they also cite.

There isn't a fundamental breakthrough or innovation here that was patently impossible before, or that they are lying about: this combines prior work, iterates upon it, and scales it up.

[0]: https://arxiv.org/abs/2304.03442

[1]: https://voyager.minedojo.org/

Re: Project Sid: Many-agent simulations toward AI civilization

#76

Post by caetris2 »

bob1029 wrote: post #70

I feel like there is some kind of information theory constraint which confounds our ability to extract higher order behavior from multiple instances of the same LLM. I spent quite a bit of time building a multi agent simulation last year and wound up at the same conclusion every day - this is all just a roundabout form of prompt engineering. Perhaps it is useful as a mental model, but you can flatten the whole thing…

You've absolutely nailed it here, I agree. To make any progress at all at the tremendously difficult problem they are trying to solve, they need to be frank about just how far away they are from what it is they are marketing.

I am whole-heartedly in support of commercial interests to drum of awareness and engagement by the authors. This is definitely a cool thing to be working on, however, what does make more sense is to frame the situation more honestly and attract folks to the desire of solving tremendously hard problems based on a level of expertise and awareness that truly moves the ball forward.

What would be far more interesting would be for the folks involved to say all the ten thousand things that went wrong in their experiments and to lay out the common-sense conclusions from those findings (just like the one you shared, which is truly insightful and correct).

We need to move past this industry and their enablers that continually try to win using the wrong methodology -- pushing away the most inventive and innovative people that are ripe and ready to make paradigm shifts in the AI field and industry.

Re: Project Sid: Many-agent simulations toward AI civilization

#77

Post by caetris2 »

afro88 wrote: post #72
caetris2 wrote: post #52

Earlier quoted context omitted.

LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here. Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false. The ideas here are not supported by any…

Perhaps I've made a big assumption / oversimplification about how this works. But.. > LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here Yes. I never said they were stateful? The context given is the state. And training data is hugely important. Once upon a time there was a guy that claimed ChatGPT could simulate a command line shell. "Si…

These are extremely hard problems to solve and it is important for any claims to be validated at this early phase of generative AI.

Re: Project Sid: Many-agent simulations toward AI civilization

#78

Post by wombatpm »

hackathonguy wrote: post #65

I'm curious if it might be possible that an AI "civilization", similar to the one proposed by Altera, could end up being a better paradigm for AGI than a single LLM, if a workable reward system for the entire civilization was put in place. Meaning, suppose this AI civilization was striving to maximize [scientific_output] or [code_quality] or any other eval, similar to how modern countries try to maximize GDP - would…

Paperclip production?

Re: Project Sid: Many-agent simulations toward AI civilization

#79

Post by Philpax »

Reubend wrote: post #71
caetris2 wrote: post #38

I've reviewed the paper and I'm confident this paper was fabricated over a collection of false claims. The claims made are not genuine and should not be taken at face value without peer review. The provided charts and graphics are sophisticated forgeries in many cases when reviewing and vetting their applicability to the claims made. It is currently not possible for any kind of LLM to do what is being proposed, while…

Yeah, I haven't looked into this much so far but I am extremely skeptical of the claims being made here. For one agent to become a tax collector and another to challenge the tax regime without such behavior being hard coded would be extremely impressive.

They were assigned roles to examine the spread of information and behaviour. The agents pay tax into a chest, as decreed by the (dynamic) rules. There are agents assigned to the roles of pro- and anti-tax influencers; agents in proximity to these influencers would change their own behaviour appropriately, including voting for changes in the tax.

So yes, they didn't take on these roles organically, but no, they weren't aiming to do so: they were examining behavioral influence and community dynamics with that particular experiment.

I'd recommend skimming over the paper; it's a pretty quick read and they aren't making any truly outrageous claims IMO.

Re: Project Sid: Many-agent simulations toward AI civilization

#80

Post by teaearlgraycold »

caetris2 wrote: post #76
bob1029 wrote: post #70

I feel like there is some kind of information theory constraint which confounds our ability to extract higher order behavior from multiple instances of the same LLM. I spent quite a bit of time building a multi agent simulation last year and wound up at the same conclusion every day - this is all just a roundabout form of prompt engineering. Perhaps it is useful as a mental model, but you can flatten the whole thing…

You've absolutely nailed it here, I agree. To make any progress at all at the tremendously difficult problem they are trying to solve, they need to be frank about just how far away they are from what it is they are marketing. I am whole-heartedly in support of commercial interests to drum of awareness and engagement by the authors. This is definitely a cool thing to be working on, however, what does make more sense i…

It would however be very interesting to see these kinds of agents in a commercial video game. Yes they are shallow in their perception of the game world. But they’re a big step up from the status quo.
Post reply on HN