Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Sid: Many-agent simulations toward AI civilization

github.com

Post reply on HN
Print view View original

51–60 of 173 posts

Re: Project Sid: Many-agent simulations toward AI civilization

#51

Post by aithrowawaycomm »

NoboruWataya wrote: post #12

This seems very cool - I am sceptical of the supposed benefits for "civilization" but it could at least make for some very interesting sim games. (So maybe it will be good for Civilization moreso than civilization.)

I think the Firaxis Civilization needs a cheap AlphaZero AI rather than an LLM: there are too many dumb footguns in Civ to economically hard-code a good strategic AI, yet solving the problem by making the enemies cheat is plain frustrating. It would be interesting to let an ANN play against a "classical" AI until it consistently beats each difficulty level, building a hierarchy. I am sure someone has already looked into this but I couldn't find any sources.

I am a bit skeptical about how computationally expensive a very crappy Civ ANN would be to run at inference time, though I actually have no idea how that scales - it hardly needs to be a grandmaster, but the distribution of dumb mistakes has a long tail.

Also, the DeepMind Starcraft 2 AI is different from AlphaZero since Starcraft is not a perfect information game. The AI requires a database of human games to "get off the ground"; otherwise it would just get crushed over and over in the early game, having no idea what the opponent is doing. It's hard to get that training data with a brand new game. Likewise Civ has always been a bit more focused on artistic expression than other 4x strategy games; maybe having to retrain an AI for every new Wonder is just too much of a burden.

Re: Project Sid: Many-agent simulations toward AI civilization

#52

Post by caetris2 »

afro88 wrote: post #43
caetris2 wrote: post #38

I've reviewed the paper and I'm confident this paper was fabricated over a collection of false claims. The claims made are not genuine and should not be taken at face value without peer review. The provided charts and graphics are sophisticated forgeries in many cases when reviewing and vetting their applicability to the claims made. It is currently not possible for any kind of LLM to do what is being proposed, while…

> this paper seems indicate that election-related activities were coordinated by groups of AI agents in a simulation I mean, that's surely within the training data of LLMs? The effectiveness etc of the election activities is likely very low. But I don't think it's outside the realms of possibility that the agents prompted each other into the latent spaces of the LLM to do with elections.

LLMs are stateless and they do not remember the past (as in they don't have a database), making the training data a non-issue here. Therefore, the claims made here in this paper are not possible because the simulation would require each agent to have a memory context larger than any available LLM's context window. The claims made here by the original poster are patently false.

The ideas here are not supported by any kind of validated understanding of the limitations of language models. I want to be clear -- the kind of AI that is being purported to be used in the paper is something that has been in video games for over 2 decades, which is akin to Starcraft or Diablo's NPCs.

The key issue is that this is a intentional false claim that can certainly damage mainstream understanding of LLM safety and what is possible at the current state of the art.

Agentic systems are not well-suited to achieve any of the things that are proposed in the paper, and Generative AI does not enable these kinds of advancements.

Re: Project Sid: Many-agent simulations toward AI civilization

#53

Post by shagie »

airstrike wrote: post #6

I've thought about this a lot. I'm no philosopher or AI researcher, so I'm just spitballing... but if I were to try my hand at it, I think I'd like to start from "principles" and let systems evolve or at least be discoverable over time Principles would be things like self-preservation, food, shelter and procreating, communication and memory through a risk-reward calculation prism. Maybe establishing what is "known" v…

Evolve is another beast... but for the: "I've thought about this a lot. I'm no philosopher or AI researcher, so I'm just spitballing... but if I were to try my hand at it, I think I'd like to start from "principles" and let systems evolve or at least be discoverable over time" part, hunt up a copy of "The Society of Mind" by Minsky who was both and wrote about that idea.

https://en.wikipedia.org/wiki/Society_of_Mind

> The work, which first appeared in 1986, was the first comprehensive description of Minsky's "society of mind" theory, which he began developing in the early 1970s. It is composed of 270 self-contained essays which are divided into 30 general chapters. The book was also made into a CD-ROM version.

> In the process of explaining the society of mind, Minsky introduces a wide range of ideas and concepts. He develops theories about how processes such as language, memory, and learning work, and also covers concepts such as consciousness, the sense of self, and free will; because of this, many view The Society of Mind as a work of philosophy.

> The book was not written to prove anything specific about AI or cognitive science, and does not reference physical brain structures. Instead, it is a collection of ideas about how the mind and thinking work on the conceptual level.

Its very approachable as a layperson in that part of the field of AI.

Re: Project Sid: Many-agent simulations toward AI civilization

#54

Post by mistermann »

catlifeonmars wrote: post #29

This looks like it is a really cool toy. It does not strike me as particularly useful from a scientific research perspective. There does not appear to be much thought put into experimental design and really no clear objectives. Is the bar really this low for academic research these days?

The scientific method has utility, but it's not a pre-requisite for utility.

Some people prefer speed and the uncertainty that comes with it.

Re: Project Sid: Many-agent simulations toward AI civilization

#55

Post by gmuslera »

ilbeeper wrote: post #46
gmuslera wrote: post #30

They probably will fall fast into tragedy of the commons kind of situations. We developed most of our civilization while there was enough room for growing and big decisions were centralized, and started to get into bad troubles when things became global enough. With AIs some of those "protections" may not be there. And hardcoding strategies to avoid this may already put a limit on what we are simulating.

> We developed most of our civilization while there was enough room for growing and big decisions were centralized, and started to get into bad troubles when things became global enough. Citation needed. But even if I will get on board with you on that, wouldn't it be to start developing for global scale right from the start, instead of starting in small local islands and then try to rework that into global ecosystem…

The problem with emulations is human patience. If you don't need/have human interaction this may run pretty fast. And at the end, what matter is how sustainable it is in the long run.

Re: Project Sid: Many-agent simulations toward AI civilization

#56

Post by foota »

NoboruWataya wrote: post #12

This seems very cool - I am sceptical of the supposed benefits for "civilization" but it could at least make for some very interesting sim games. (So maybe it will be good for Civilization moreso than civilization.)

I'm somewhat amazed that companies releasing strategy games aren't using AI to test out different cards and what not to find broken things before release (looking at you, Hearthstone)

Re: Project Sid: Many-agent simulations toward AI civilization

#57

Post by whatshisface »

isoprophlex wrote: post #3

Now these seem to be truly artificially intelligent agents. Memory, volition, autonomy, something like an OODA loop or whatever you want to call it, and a persistent environment. Very nice concept, and I'm positive the learnings can be applied to more mundane business problems, too. If only I could get management to understand that a bunch of prompts shitting into eachother isn't "cutting-edge agentic AI"... But then…

Just so you know, the English noun for things that have been learned is, "lessons."

Re: Project Sid: Many-agent simulations toward AI civilization

#59

Post by airstrike »

fragmede wrote: post #49
airstrike wrote: post #6

I've thought about this a lot. I'm no philosopher or AI researcher, so I'm just spitballing... but if I were to try my hand at it, I think I'd like to start from "principles" and let systems evolve or at least be discoverable over time Principles would be things like self-preservation, food, shelter and procreating, communication and memory through a risk-reward calculation prism. Maybe establishing what is "known" v…

So a modernized version of Spore.

Basically what we all wished Spore had been ;-)

Re: Project Sid: Many-agent simulations toward AI civilization

#60

Post by mindcrime »

whatshisface wrote: post #57
isoprophlex wrote: post #3

Now these seem to be truly artificially intelligent agents. Memory, volition, autonomy, something like an OODA loop or whatever you want to call it, and a persistent environment. Very nice concept, and I'm positive the learnings can be applied to more mundane business problems, too. If only I could get management to understand that a bunch of prompts shitting into eachother isn't "cutting-edge agentic AI"... But then…

Just so you know, the English noun for things that have been learned is, "lessons."

Also: "learnings".

https://dictionary.cambridge.org/us/dictionary/english/learn...

"knowledge or a piece of information obtained by study or experience"

"I am already incorporating some of these learnings into my work and getting better results."

Post reply on HN