Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can a MUD evaluate LLMs? A $99 proof of concept

cruciblebench.ai

Post reply on HN
Print view View original

71–80 of 88 posts

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#71

Post by iepathos »

outofpaper wrote: post #52
iepathos wrote: post #20

Earlier quoted context omitted.

MUDs never disappeared. People stopped finding them engaging enough to use. Blaming Discord reverses the causality. MUDs lost engagement long before disco was created. People congregate on disco because it offers a more compelling social interface, even if it lacks the depth of a persistent game world. The opportunity I'd draw from this isn’t to convince people to return to older and less engaging interfaces. It’s to…

Is your autocomplete passing you disco instead of Discord and you're just going with it? Or are you implying that before the creation of disco music people had grown tired of MUDs?

Haha, ty for this. That's just a personal habit. I tend to call Discord disco for short amongst my friends.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#72

Post by pigpop »

munk-a wrote: post #39
tracker1 wrote: post #31

Earlier quoted context omitted.

Would be kind of a cool experiment for a mud-style interface, where each npc/avatar was backed by a lower-cost LLM... similar rules for the npcs in the game, but a background feed, and history of interactions with other players as background. Maybe limiting npc's to only a certain number of moves that aren't a response to other users per day... It could be a lot of fun.

A MUD exists as a platform for creative writing - introducing LLMs into one would deprive it of its primary function.

In some cases but coming from someone who played MUDs back in the day on a monochrome display and dial-up I know I would have been very interested in playing one with NPCs powered by an LLM had they been available back then. The NPCs as they existed were generally just furniture and the interesting part was playing with and talking to other people.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#73

Post by kingbirdy »

munk-a wrote: post #55
kingbirdy wrote: post #49

Earlier quoted context omitted.

> There is an ongoing project doing this (that launches this weekend I believe) to revive a MUD called Shadows of Isildur Interesting, could you share more info? I couldn't find anything on Google. I played a fair amount of SOI back in the day and it'd be fun to start over on a fresh server.

play.shadowsofisildur.com:4500 - if you'd like (it isn't on the old domain anymore) there's also a discord that I'm a bit hesitant to link to where all the discussion is taking place that has a lot of the old folks hanging around and nostalgifying.

So the website doesn't work and the discord is private for no reason? Unfortunately this is reminding me why I stopped playing MUDs :( For a genre that's entirely dependent on keeping up an established and invested user base, the average MUD sure seems to dislike its players, especially new ones.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#75

Post by Davisb135 »

aphexairlines wrote: post #69

It's interesting that you chose to measure how well each LLM did in talking to other NPCs, and having each NPC also use an LLM to react to each input. Why not have the LLM fight NPCs and loot items (both a significant part of the MUD experience, neither requiring LLMs on the server side), then measure character progression in experience and stats?

Just starting small. Those are all ideas for Phases 2 & 3.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#78

Post by gverrilla »

I tried MUD a few times as a teenager, but for some reason I didn't like it. I prefered playing RPG on a long dead phpBB forum (LdC), and to this day I remember the very simple mechanic (for both players and gamemaster): an asterisk for character speech, and (#) for everything else.

A single post might look like this:

    # By hearing those absurdities, memories rush to the mind
    of Vicent Panclast — memories he'd like to avoid. He gets
    physically agitated and raises his pistol high in the air.
    * They shall not pass! We must do anything we can to stop
    these mf nazis!!
From what I remember there was no dice at all — the gm just did whatever he'd like. Your character sheet was just a story of your character plus some description. We had a fantastic gamemaster, so it made for great rpg. Good times.

I wonder what something like that could look like with the right use of AI, today. I'd like to think kids are exploring this kind of thing nowadays - I hope they are, because it's a ton of fun (even without AI, kids).

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#79

Post by munk-a »

kingbirdy wrote: post #73
munk-a wrote: post #55

Earlier quoted context omitted.

play.shadowsofisildur.com:4500 - if you'd like (it isn't on the old domain anymore) there's also a discord that I'm a bit hesitant to link to where all the discussion is taking place that has a lot of the old folks hanging around and nostalgifying.

So the website doesn't work and the discord is private for no reason? Unfortunately this is reminding me why I stopped playing MUDs :( For a genre that's entirely dependent on keeping up an established and invested user base, the average MUD sure seems to dislike its players, especially new ones.

The website is down right now and the discord isn't private but I'm hesitant to personally publish it in such a public place.

Re: Can a MUD evaluate LLMs? A $99 proof of concept

#80

Post by Varelion »

CTDOCodebases wrote: post #57
Varelion wrote: post #6

Earlier quoted context omitted.

I honestly worry that the relevance of text-based systems is largely tied with literacy and how it interacts with the player's imagination -- something largely in decline, at least in the US.

It's not just imagination the relevance is also tied to focus and attention. I have multiple friends who claim that the reason they don't read is because are "visual learners" . The real truth is they have a 30 second attention span and can't finish a page without daydreaming so in order to stay focussed they need a YouTube video than contains the same information but with constant visual changes.

Jesus H. Christ, never thought about that.
Post reply on HN