Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Sid: Many-agent simulations toward AI civilization

github.com

Post reply on HN
Print view View original

131–140 of 173 posts

Re: Project Sid: Many-agent simulations toward AI civilization

#131

Post by MichaelZuo »

chefandy wrote: post #130
bob1029 wrote: post #70

I feel like there is some kind of information theory constraint which confounds our ability to extract higher order behavior from multiple instances of the same LLM. I spent quite a bit of time building a multi agent simulation last year and wound up at the same conclusion every day - this is all just a roundabout form of prompt engineering. Perhaps it is useful as a mental model, but you can flatten the whole thing…

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyon…

Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.

Re: Project Sid: Many-agent simulations toward AI civilization

#132

Post by 8338550bff96 »

grugagag wrote: post #120
8338550bff96 wrote: post #116

Earlier quoted context omitted.

How do you distinguish between the real thing and a perfect simulation of the real thing? You seem to be engaged in faith-based reasoning at this point. If you were born in a sensory deprivation chamber you also would have no inner world, and you wouldn't have anything at all to say about solving chemistry problems. > Im actually surprised so many get fooled by the hype and are ready to declare a winner. Find me one…

It’s the approach that matters. When it gets to 99.9 percent it’s pretty good to be dangerous. At that point it would be hard to tell but not impossible. As soon as a new type problem comes out it will bork on you and need retraining. It’ll be a game of catch albeit an very inneficient one. Im sure we will find a more efficient method eventually but the point still stands, what we have isn’t it. I’ll shut up when I s…

I'll shut up when I see humans get 99.9% on anything. This seems an awful lot like non-meat brain prejudice where standards that humans do not live up to at all are imposed on other things in order to be worthy of consideration

Re: Project Sid: Many-agent simulations toward AI civilization

#133

Post by chefandy »

MichaelZuo wrote: post #131
chefandy wrote: post #130

Earlier quoted context omitted.

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyon…

Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.

I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of being as compelling to socially-focused role-playing gamers as online games?

This is my field so I'm always looking for the angle that new tech will take. I still rank this lower than VR— with all of its problems— for potential to significantly change player interactions. Tooling to make games is a different story, but for actual use in games? I don't see it yet.

Re: Project Sid: Many-agent simulations toward AI civilization

#134

Post by JohnMakin »

chefandy wrote: post #130
bob1029 wrote: post #70

I feel like there is some kind of information theory constraint which confounds our ability to extract higher order behavior from multiple instances of the same LLM. I spent quite a bit of time building a multi agent simulation last year and wound up at the same conclusion every day - this is all just a roundabout form of prompt engineering. Perhaps it is useful as a mental model, but you can flatten the whole thing…

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyon…

It's been posted in-depth a few times across this forum to varying degrees by game developers - I was initially very excited about the implementation of LLM's in NPC interactions, until I read some of these posts. The gist of it was - the thing that makes a game fundamentally a game is its constraints. LLM-based NPC's fundamentally break these constraints in a way that is not testable or predictable by the developer and will inevitably destroy the gameplay experience (at least with current technology).

Re: Project Sid: Many-agent simulations toward AI civilization

#135

Post by airstrike »

BlueTemplar wrote: post #127
airstrike wrote: post #6

I've thought about this a lot. I'm no philosopher or AI researcher, so I'm just spitballing... but if I were to try my hand at it, I think I'd like to start from "principles" and let systems evolve or at least be discoverable over time Principles would be things like self-preservation, food, shelter and procreating, communication and memory through a risk-reward calculation prism. Maybe establishing what is "known" v…

Huh, so the video actually works ? It just shows up «No video with supported format and MIME type found.» for me... Yeah, memes and genes are both memory, though at different timescales.

It works on some browsers. I'm normally on Firefox but had to dust off Safari to watch it. Crazy I still have to do this in 2024...

Re: Project Sid: Many-agent simulations toward AI civilization

#136

Post by chefandy »

JohnMakin wrote: post #134
chefandy wrote: post #130

Earlier quoted context omitted.

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyon…

It's been posted in-depth a few times across this forum to varying degrees by game developers - I was initially very excited about the implementation of LLM's in NPC interactions, until I read some of these posts. The gist of it was - the thing that makes a game fundamentally a game is its constraints. LLM-based NPC's fundamentally break these constraints in a way that is not testable or predictable by the developer…

Yeah, same. Epic's Matrix demo implemented it and even without a plot, the interactions were so heavily guided that the distinction was pointless. So you can find out what that NPCs spous's name is and their favorite color. It's that neat? Sure it's neat. It's it going to make it a better game? Probably less than hiring another good writer to make NPC dialog. To be truly useful, I think they would have to be able to affect the world in meaningful ways that worked with the game plot, and again, when you clamp that down as much as you'd need to to still have a plot, you're looking at a fancy decision tree.

Re: Project Sid: Many-agent simulations toward AI civilization

#137

Post by mywittyname »

chefandy wrote: post #133
MichaelZuo wrote: post #131

Earlier quoted context omitted.

Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.

I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of…

Sandbox games are probably where they will shine. Imagine being able to play Minecraft, and tell a prompt to generate a world that resembles Tatooine, or a vampire-themed mansion. Expectations are lower with sandbox games, so there's no risk of breaking immersion like would happen with an LLM Elder Scrolls game when someone tricks in NPC into solving problems in python.

Granted, I'm certain there will be copyrights issues associated with this capability, which is why I don't think it will be established game companies who first take a crack at this approach.

Re: Project Sid: Many-agent simulations toward AI civilization

#138

Post by chefandy »

mywittyname wrote: post #137
chefandy wrote: post #133

Earlier quoted context omitted.

I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of…

Sandbox games are probably where they will shine. Imagine being able to play Minecraft, and tell a prompt to generate a world that resembles Tatooine, or a vampire-themed mansion. Expectations are lower with sandbox games, so there's no risk of breaking immersion like would happen with an LLM Elder Scrolls game when someone tricks in NPC into solving problems in python. Granted, I'm certain there will be copyrights i…

The problem is what it takes to implement that. I've seen companies currently trying to do exactly that, and their demos go like this "ok, give me a prompt for the environment" and if they're lucky, they can cherry pick some stuff the crowd says and if they're not, they sheepishly ask for a prompt that would visit indicate one of 5 environment types they've worked on and include several of the dozen premade textured meshes they've made, and in reality you've got a really really expensive procedural map with asset placement that's worse than if it was done using traditional semi-pre-baked approaches. A deceptive amount of work goes into the nitty gritty of making environments, and even with all of the incredible tooling that's around now, we are not even close to automating that. It's worth noting that my alma mater has game environment art degree programs. Unless you're making these things, you can't easily see how much finesse and artistic sensibility it takes to make beautiful compositions with complementary lighting and nice atmospheric progression. It's not just that nobody has really given it a go— it's really difficult. When you have tooling that uses AI controlled by an artist that knows these things, that's one thing. When they need to make great results every time so players keep coming back? That's a very different task. I've never met anyone that thought it was remotely currently feasible without lacking knowledge of generative AI, game development, or both.

Automating the tools so a smaller workforce can make more worlds and more possibilities? We're already there— but it's a very large leap to remove the human creative and technical intermediaries.

Re: Project Sid: Many-agent simulations toward AI civilization

#139

Post by BlueTemplar »

aithrowawaycomm wrote: post #51
NoboruWataya wrote: post #12

This seems very cool - I am sceptical of the supposed benefits for "civilization" but it could at least make for some very interesting sim games. (So maybe it will be good for Civilization moreso than civilization.)

I think the Firaxis Civilization needs a cheap AlphaZero AI rather than an LLM: there are too many dumb footguns in Civ to economically hard-code a good strategic AI, yet solving the problem by making the enemies cheat is plain frustrating. It would be interesting to let an ANN play against a "classical" AI until it consistently beats each difficulty level, building a hierarchy. I am sure someone has already looked i…

Galactic Civilizations 2 (also, 1,3,4 ??) in the same genre is well-known for its AI, good even without handicaps or cheats. This includes trading negotiations BTW.

(At least good compared to what other 4X have, and your average human player - not the top players that are the ones that tend to discuss the game online in the first place.)

EDIT : I suspect that it's not unrelated that GalCiv2 is kind of... boring as 4X go - as a result of a good AI having been a base requirement ?

Speaking of StarCraft AI... (for SC1, not 2, and predating AlphaZero by many years) :

https://arstechnica.com/gaming/2011/01/skynet-meets-the-swar...

Re: Project Sid: Many-agent simulations toward AI civilization

#140

Post by grugagag »

8338550bff96 wrote: post #132
grugagag wrote: post #120

Earlier quoted context omitted.

It’s the approach that matters. When it gets to 99.9 percent it’s pretty good to be dangerous. At that point it would be hard to tell but not impossible. As soon as a new type problem comes out it will bork on you and need retraining. It’ll be a game of catch albeit an very inneficient one. Im sure we will find a more efficient method eventually but the point still stands, what we have isn’t it. I’ll shut up when I s…

I'll shut up when I see humans get 99.9% on anything. This seems an awful lot like non-meat brain prejudice where standards that humans do not live up to at all are imposed on other things in order to be worthy of consideration

That’s actually good. The more voices the better, that will make for a more vibrant discussion.
Post reply on HN