Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM based agents as Dungeon Masters

studenttheses.uu.nl

Post reply on HN
Print view View original

91–100 of 144 posts

Re: LLM based agents as Dungeon Masters

#91

Post by TeMPOraL »

giantrobot wrote: post #76
TeMPOraL wrote: post #56

Earlier quoted context omitted.

Two reasons: 1) "LLM DM" isn't merely mean substituting for human DM in a D&D session. The same capability can be used as a component in a video game, to breathe life into the game world, have it interactively react and evolve along with the player. EDIT: Take a game like Rimworld, that relies on a scripted RNG dubbed a "storyteller", to decide what random events to hit you with and how hard. It's fun early on, but i…

> An LLM DM is exactly what could add that story, make overcoming challenges feel meaningful and allow for player decisions to actually impact the world deeply. No it's not. I don't think you're going to find an LLM with a large enough context window to have a meaningfully involving story spanning multiple sessions. An LLM isn't going to craft a story element tailored to a character, or more importantly, an individua…

> I don't think you're going to find an LLM with a large enough context window to have a meaningfully involving story spanning multiple sessions.

Sure you will.

> An LLM isn't going to craft a story element tailored to a character, or more importantly, an individual player.

Sure it is.

> An LLM also doesn't really understand the game rules and isn't going to be able to adjudicate house rules based on fun factor.

Sure it will.

You need to use the tools for their purpose, not for the opposite of it. LLMs have finite context, you need to manage it. LLMs don't have a built-in loop, you need to supply it.

Character stats, names, details about players - those are inputs, and structured ones at that. LLMs shouldn't store them - that's what storage media are for, whether in-memory or a database or a piece of paper. Nor should they manipulate them directly - that's what game systems are for, whether implemented in code or in a rulebook run on a human DM. LLMs are to make decisions - local, intuitive decisions, based on what is in their context. That could be deciding what a character says in a given situation. Or how to continue the story based on worldbuilding database. Or how to update the worldbuilding database based on what it just added to the story. Etc.

Re: LLM based agents as Dungeon Masters

#92

Post by TeMPOraL »

mlyle wrote: post #83
giantrobot wrote: post #76

Earlier quoted context omitted.

> An LLM DM is exactly what could add that story, make overcoming challenges feel meaningful and allow for player decisions to actually impact the world deeply. No it's not. I don't think you're going to find an LLM with a large enough context window to have a meaningfully involving story spanning multiple sessions. An LLM isn't going to craft a story element tailored to a character, or more importantly, an individua…

His example: Rimworld is a great universe where we think about characters' stories, and there's really just like, a couple dozen attributes and straightforward ways they interact with the game. An LLM context window could easily have 20 times as much interpersonal state, and make it interact in much more unexpected (but plausible) ways. That's going to be a surprising and rewarding gaming experience once someone figu…

Context window is arbitrary and can be adjusted on demand. When a new random event needs to be generated, and then fleshed out, the context can contain e.g. list of facts about the story so far, the overall story arc, summary of main characters or plot threads. This can be used by LLM to decide e.g. which faction will attack, and who will be on it, and what their goal will be, etc. After the event concludes, it just becomes another line in story event history. Meanwhile, that can be fed to a differently prompted LLM to evolve the plot arc, update motivations of background characters, etc.

I have a feeling people imagine LLMs as end-to-end DMs that should somehow remember everything and do everything in one inference round. That's not what they are. They're building blocks. They're to be chained and mixed with classical flow control, algorithms, and data storage (as well as the whole interactive game mechanics, in videogame context).

Re: LLM based agents as Dungeon Masters

#93

Post by bongodongobob »

drak0n1c wrote: post #88

For those averse to LLM defeating the social aspect of tabletop roleplaying, perhaps a better entry role for an LLM agent is as NPC party companions/followers. Set up the agent with a detailed character sheet and hidden backstory/motivations and set it to listen in on conversations or VTT chat, and interject dialog on a non-spammy basis. Could lead to unusual situational ideas or at least humorous moments.

This is what our DM does. It works really well for NPCs and saves him a shit ton of time and effort.

Re: LLM based agents as Dungeon Masters

#94

Post by gwern »

giantrobot wrote: post #76
TeMPOraL wrote: post #56

Earlier quoted context omitted.

Two reasons: 1) "LLM DM" isn't merely mean substituting for human DM in a D&D session. The same capability can be used as a component in a video game, to breathe life into the game world, have it interactively react and evolve along with the player. EDIT: Take a game like Rimworld, that relies on a scripted RNG dubbed a "storyteller", to decide what random events to hit you with and how hard. It's fun early on, but i…

> An LLM DM is exactly what could add that story, make overcoming challenges feel meaningful and allow for player decisions to actually impact the world deeply. No it's not. I don't think you're going to find an LLM with a large enough context window to have a meaningfully involving story spanning multiple sessions. An LLM isn't going to craft a story element tailored to a character, or more importantly, an individua…

> I don't think you're going to find an LLM with a large enough context window to have a meaningfully involving story spanning multiple sessions.

First, you don't need much of any context window because you can finetune the LLM. Don't mistake specific engineering choices and tradeoffs and deployment convenience for intrinsic limitations of the technology.

Second, LLMs like Gemini now have context windows of millions of tokens, corresponding to millions of words. Seems like enough for 'multiple sessions'.

> An LLM isn't going to craft a story element tailored to a character, or more importantly, an individual player. It's not going to understand Sam couldn't make last week's session. An LLM also doesn't really understand the game rules and isn't going to be able to adjudicate house rules based on fun factor.

An LLM can do all of that, and you definitely do not know that they can't.

> They'll be no better a game master than a MadLibs book.

They've been better than a Madlibs book since AI Dungeon 1 which was like 6 years ago.

Re: LLM based agents as Dungeon Masters

#95

Post by bongodongobob »

devmor wrote: post #69

Like others, I don't really understand why anyone would desire this. D&D is a social activity, why would I want to remove the most core human component of a social activity. It'd be like someone who enjoys going to a sports bar with friends deciding they'd rather sit next to a vending machine.

Do you take issue with Baldur's Gate 3? Video games in general? Why not?

Re: LLM based agents as Dungeon Masters

#96

Post by HanClinto »

This is extremely interesting to me. I understand why so many people are surprised that people would want this, but for whatever reason, I'm very much in the target audience.

I'm a "perpetual DM" in most of my roleplaying groups, having led campaigns spanning years, but very rarely do I get to participate as a player. My kids love me running adventures and one-shots, and while I enjoy it, people don't always recognize how much work it is -- both the prep, as well as the mental effort of being "on" for the entire duration of the adventure. I like afternoon one-shot adventures, but I dislike reading through long sourcebooks. I enjoy crafting puzzles and presenting challenges to the players, but also recognize the mental effort that this takes, and sometimes I just want a copilot to handle the lore and the setting. I already rely on the rules-nerds in my group to act as copilots for the specifics of game mechanics -- I don't have time to read through that many tables and commit them to memory.

Re: datasets, the paper authors used a dataset that they created by scraping Critical Role transcripts -- I doubt they had permission for that. I like contributing to open datasets for training open-source LLMs, and I think it would be really sweet for a playgroup to be able to contribute their play transcripts for such a thing. I would love to see a standard open-source dataset be collected for this, and perhaps even a standardized benchmark for quantifying the performance of particular LLMs in RPG contexts -- rules compliance, needle-in-haystack fact finding, consistency of roleplaying, understanding user intent, challenge creation / resolution, etc.

Re: LLM based agents as Dungeon Masters

#97

Post by mlyle »

TeMPOraL wrote: post #92
mlyle wrote: post #83

Earlier quoted context omitted.

His example: Rimworld is a great universe where we think about characters' stories, and there's really just like, a couple dozen attributes and straightforward ways they interact with the game. An LLM context window could easily have 20 times as much interpersonal state, and make it interact in much more unexpected (but plausible) ways. That's going to be a surprising and rewarding gaming experience once someone figu…

Context window is arbitrary and can be adjusted on demand. When a new random event needs to be generated, and then fleshed out, the context can contain e.g. list of facts about the story so far, the overall story arc, summary of main characters or plot threads. This can be used by LLM to decide e.g. which faction will attack, and who will be on it, and what their goal will be, etc. After the event concludes, it just…

Maybe. But for something like Rimworld++++, you don't need all that sophistication. You need a pile of relevant facts and a clear task to determine something about the game or write some text for the user. Sure, curating and retrieving selectively would probably be even better and more flexible.

Re: LLM based agents as Dungeon Masters

#98

Post by johnfn »

2099miles wrote: post #3

This is so interesting that people want this. This is beyond the guy saying “people don’t like making music so we made ai so it’s easier and faster to just prompt it to make music” . Like in my experience the best part of d&d is the dungeon master who knows their friends making a story and details specifically for fun and the fun of the group. And the interactions between players and the dm is where I derived the mos…

This is nothing new. Bringing ideas from DND into video games is as old as video games themselves. In interviews, John Carmack talks often about how he used to play DND with his friends and then try to take that experience into the games he made. And it goes back even further than that - Rogue, the primogenitor of all roguelike games, was inspired by DND as well. I could find a bunch more examples, but you get the idea.

Re: LLM based agents as Dungeon Masters

#99

Post by int_19h »

JohnMakin wrote: post #30
pmichaud wrote: post #24

Earlier quoted context omitted.

They are a step on the road to fully responsive rpg worlds though, which don’t have to be text based.

It's been written at length here and elsewhere by game devs, but this isn't a thing that anyone would truly want. A purely AI generated or controlled world would have no constraints, and be fundamentally untestable - games aren't really games unless they have constraints. Even the 'purest' sandboxes have some kind of constraint buried within them, and I think you'd find an RPG of this type extremely boring, at least…

There's no reason why an AI-driven sandbox cannot have constraints, as well.

Now it's true that, with the current crop of LLMs, a persistent enough player would always be able to break through them. But if it takes conscious and deliberate effort, I think it's reasonable to say that whatever experience the person gets as a result, they were asking for it.

Re: LLM based agents as Dungeon Masters

#100

Post by int_19h »

cdiamand wrote: post #16

Last year, I built a DM for myself using the OpenAI api and an elevenLabs voice generator. I asked it set my character in Baldur's Gate, so it could pull upon the huge amount of DnD source material it had been trained on. A few takeaways: 1. An LLM based DM can give the player essentially infinite richness and description on anything they ask for. 2. There is difficulty in setting the rules for the LLM to follow that…

> Also, I found the LLM to be too pliable as a DM. I kept getting my way, or getting my hand held through scenarios. Maybe this is a feature?

LLMs are fine-tuned to be "helpful assistants", so they're basically sycophantic.

Post reply on HN