Speaking of MUD, has anyone tried attaching an AI to the Hitchhikers MUD[1]? [1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...
I mean that's an Infocom game, so a text adventure a la Zork.
Can a MUD evaluate LLMs? A $99 proof of concept
61–70 of 88 posts
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#62"We did not choose a MUD because it is charming. We chose it because its constraints make behavior measurable." I so fucking hate these LLMisms
[dead]
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#63Speaking of MUD, has anyone tried attaching an AI to the Hitchhikers MUD[1]? [1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...
Not Hitchhiker's specifically, we'd considered another existing one but as soon as we started digging into IP law related to MUDs, we decided if we're going to do it, we'll just start our own from scratch.
> we decided if we're going to do it, we'll just start our own from scratch
And because scripts and information about famous text based adventure games can be part of what the LLM ingested during training (so unusable for fresh reasoning).
Anyway: very, very, very, very good idea.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#64Earlier quoted context omitted.
Would be kind of a cool experiment for a mud-style interface, where each npc/avatar was backed by a lower-cost LLM... similar rules for the npcs in the game, but a background feed, and history of interactions with other players as background. Maybe limiting npc's to only a certain number of moves that aren't a response to other users per day... It could be a lot of fun.
A MUD exists as a platform for creative writing - introducing LLMs into one would deprive it of its primary function.
That sounds very different than my memory of MUDs. The ones I remember were much more a text-based version of "World of Warcraft" or other similar MMOs where you navigate through a world, complete quests, etc. You could chat with other players but it was more about teaming up for quests than a creative writing/roleplaying exercise.
Not to say that what you're describing didn't exist, but I don't think it was defining feature of all MUDs.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#65Speaking of MUD, has anyone tried attaching an AI to the Hitchhikers MUD[1]? [1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...
Speaking of Douglas Adams, Starship Titanic would be the perfect game to try reimplementing with a modern LLM. It already runs on a primitive chat AI system.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#66Earlier quoted context omitted.
A MUD exists as a platform for creative writing - introducing LLMs into one would deprive it of its primary function.
That sounds very different than my memory of MUDs. The ones I remember were much more a text-based version of "World of Warcraft" or other similar MMOs where you navigate through a world, complete quests, etc. You could chat with other players but it was more about teaming up for quests than a creative writing/roleplaying exercise. Not to say that what you're describing didn't exist, but I don't think it was defining…
There was a distinct split between people there for the grind and people there for the RP. I was there for the RP and I was mostly on MUCKs, MUSHes, and the occasional MOOs (multi-user chat kingdoms, multi-user shared hallucinations, and mud (object-oriented) ). They all had a combat system lurking somewhere in their code but those routines rarely got used.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#67Earlier quoted context omitted.
That sounds very different than my memory of MUDs. The ones I remember were much more a text-based version of "World of Warcraft" or other similar MMOs where you navigate through a world, complete quests, etc. You could chat with other players but it was more about teaming up for quests than a creative writing/roleplaying exercise. Not to say that what you're describing didn't exist, but I don't think it was defining…
There was a distinct split between people there for the grind and people there for the RP. I was there for the RP and I was mostly on MUCKs, MUSHes, and the occasional MOOs (multi-user chat kingdoms, multi-user shared hallucinations, and mud (object-oriented) ). They all had a combat system lurking somewhere in their code but those routines rarely got used.
Firstly, "multi-user chat kingdom" is a backronym that was not envisioned by the original authors of TinyMUCK, but perhaps by the users or developers of some future fork. TinyMUCK is notable for having a "Fuzzball" edition or fork, that powered FurryMUCK itself, as well as related worlds which hosted and catered to people in furry fandom.
Secondly, exactly none of these had a "combat system" of any kind. These were all derived from TinyMUD, written by Jim Aspnes at CMU. https://en.wikipedia.org/wiki/James_Aspnes
TinyMUD was a "social MUD" and its explicit goals were to be a glorified chat room, and a sort of "blank slate" for people interested in building their own extensible worlds, based on a pure textual interface. There was absolutely no "combat system" at all--in stark contrast to the other "MUDs" on the scene in those days, such as AberMUD, DikuMUD, and LPmud, which were arguably way more popular, way more violent, and way more into the "combat" or "PvP" or also PvE type play styles.
There was, of course, a "kill" command to allow one player to kill another. But the results of the kill were quite nerfed: the player was sent to their $HOME, and collected an "insurance policy" of pennies, which was partly based on the killer-determined "cost" to kill them in the first place. This wasn't combat by any means--basically, a way to sort of put an exclamation mark on a conversation.
MOO, though it was not derived from any Tiny* codebase, was also one of those "talkie" muds at its heart: it was an object-oriented, fully-programmable, extensible world-building server, where indeed, any combat system could've been implemented, if the players/programmers of a certain world agreed to include combat. But MOO was unique in giving a "barebones" server to program from first principles. So, no, none of these had combat systems, and that's their chief appeal.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#68I am having a hell of a lot of fun letting agents play and understand a (still alive, human populated) MUD, which I have also played for the last 30 years.
It’s mostly my way to play with local llm inference (m5 64gb, gwen3.6 27).
It’s amazing. They build maps, classify events (building a grammar for a parser), run experiments (to verify the grammar). They are now (given the correct tools/infrastructure) trying to fine-train a 3b model for fighting (where you need a decision for 5 seconds rounds). Basically autonomously!
Overall, a MUD does prove a great constrained sandbox for them to play in.
What started as an experiment to test local inference landed in a sweet spot for seeing models strength/weaknesses/tradeoffs. And it’s really fun.
Only problem is that Claude gets really jealous when I ask him to code their po harness running local inference. Weird world.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#69It's interesting that you chose to measure how well each LLM did in talking to other NPCs, and having each NPC also use an LLM to react to each input. Why not have the LLM fight NPCs and loot items (both a significant part of the MUD experience, neither requiring LLMs on the server side), then measure character progression in experience and stats?
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#70I am having a hell of a lot of fun letting agents play and understand a (still alive, human populated) MUD, which I have also played for the last 30 years. It’s mostly my way to play with local llm inference (m5 64gb, gwen3.6 27). It’s amazing. They build maps, classify events (building a grammar for a parser), run experiments (to verify the grammar). They are now (given the correct tools/infrastructure) trying to fi…
How is the agent interfacing with it? Are you just manually copy pasting game output and doing the response or something more integrated?