LLM based agents as Dungeon Masters

I tend to agree. I used to run Call of Cthulhu in high school, and there was a lot of preparation involved in running a good scenario. Some people are good at winging it and making stuff up as they go along. I never was. The best experiences for me and my players was always when I'd meticulously designed my own scenario, or used a store-bought scenario and read it thoroughly. Either way, hours of prep.

I would love to have an AI as an assistant Dungeon Master (or game master, or Keeper, or what have you). That is, one person in a group of players maintains the role of a master storyteller, but the AI is ready to fill in details or suggest ways to get the players back on track. This would probably be tedious if you're interacting with the LLM entirely through text, and having to manually keep it up to date with the story. But it could work well if you have a model that understands spoken language listening in on the game and generating cool images and making private suggestions to the game master.

Last year, during a period of recovery, I hosted a ChatGPT game with my nieces and newphew. I played the role of being a bride between them and the LLM. I read aloud the text, and translated their actions into queries.

With slight nudges, GPT performed well. It excelled at NPC dialog, location description, and to some extent improvisation. However, eventually the campaign fell apart on session four or five. There was a disagreement between the players over exactly how many maps we had acquired. As you know, answering this question is a needle in a haystack problem and asking the LLM proved to illicit conflicting responses. It ultimately led to me concluding that pausing the campaign was the right choice and accepting that we all still had an amazing time up to that point.

Which leads me back to the questions in the paper: > We believe that with the above evaluation methods we can try to answer the following questions: • How consistent is a LLM Agent in its generative dialogue? • How well can a LLM Agent keep a user engaged in a narrative plot? • How creative is a LLM Agent when generating complex story driven narratives?

None of these are exactly relevant to my personal experience running a TTRPG game using ChatGPT and what makes it succeed or fail. What would make such a system an improvement is being able to store and retrieve facts about players (HP, stats, inventory), NPCs(names, location, history), and Places(items, descriptions, inhabitants) in order to solve the needs-haystack problem. Creating a world ontology and being able to CRUD facts against such an ontology would be a game changer(literally).

Despite the ultimate demise of the game, the experiment was a success. I'd highly encourage anyone with a laptop, a handful of dice, and a group of interested friends to give a shot. It's fun, memorable, and a unique way to pass the time and bond.

I made something like this to let people explore the lore of my game. It has an AI act like a dungeon master as you play through a story. You can try it for free at: https://orbsccg.com/adventures

The main game is a strategic collectible card game, but the “adventure mode” is just for fun AI-powered stories.

> Care to make any citations yourself here?

Happily. I know you are wrong on the "nobody wants this" statement because I want it. With a sweeping generic statement like "nobody wants this" a single example is enough to disprove it. There you have it.

> what advances are being made in AI technology in this gaming that lends you such confidence?

There is a ton of experimenting going on. AI Dungeon and Deep Realms are the two obvious ones. I don't think anyone has found the golden solution yet, but that is also not evidence that no such thing exists.

I largely see it as the digital next step up from something like the Mythic Game Master Emulator, which is popular for solo play and things like trying out new systems.

And there's quite a lot of us that like to play but because of life commitments, getting together with a group on a regular schedule is difficult.