Live data from Hacker News

Can a MUD evaluate LLMs? A $99 proof of concept

cruciblebench.ai

21–30 of 88 posts

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#21
+1 to using MUD as a learning tool. Learn about agent loops, prompting iteration, etc.

I vibe coded a few room proof of concept MUD. I added a tools called `oracle` (aka me) that it could ask me questions to help along its way; and the ability to interject in the loop with a hint. Just as I might want an llm to stop to ask me for help instead of plodding along...

Interesting to limit the number of turns and see where it gets stuck or how quickly it can finish.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#22
post #4

A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…

MUDs never disappeared. People stopped finding them engaging enough to use. Blaming Discord reverses the causality. MUDs lost engagement long before disco was created. People congregate on disco because it offers a more compelling social interface, even if it lacks the depth of a persistent game world. The opportunity I'd draw from this isn’t to convince people to return to older and less engaging interfaces. It’s to…

I specifically outlined non-MUD role playing environments because I know MUDs lost popularity long before Discord. I am outlining that any form of tech, besides Discord, is sidelines -- MUDs and a hopeful resurgence as well.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#24
post #4

A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…

discworld mud (any many others) ase still totally worth playing

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#25

Speaking of MUD, has anyone tried attaching an AI to the Hitchhikers MUD[1]? [1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...

that's funny. the very first thing i had an llm do was to ask it to pretend to be the HHGTTG infocom game. Can't remember how it did.. not amazingly but was interesting still.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#27
I was a bit afraid that MUD was an acronym for something else, I clicked in about 10% hope that this will be for multi user dungeons and behold, thank you. Interesting read.

That brought back memories when we were trying to find free terminal in few departments at our uni to get access to one that has internet connectivity.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#28
post #4

A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…

You would probably like my project https://github.com/timbran-project/moor

That's a cool project, thanks for sharing!

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#29

"We did not choose a MUD because it is charming. We chose it because its constraints make behavior measurable." I so fucking hate these LLMisms

That's fair. We leaned on LLMs to help draft parts of the paper but disclosed it in the acknowledgements section. We didn't appreciate the amount of work this "hobby" experiment would require, and so looked for ways to help get this out the door.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#30
post #4

A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…

Technically MUDs still are a thing, there's a good number still running. Though sounds like you don't really just want a MUD where you want to go out mobbing but a more social thing (but not IRC?). I suspect MUDs may have a bit of a resurgence soon, I've seen more and more interest in them as of late simply because they seem like an excellent pairing with LLMs which easily consume text. Though I think for your needs…

Having stepped away from MUDs ourselves for years, it's been interesting to see what the community has been cooking up in terms of UI/UX, Evennia, MUDlet, TUIs, etc.
Post reply on HN