Earlier quoted context omitted.
I honestly worry that the relevance of text-based systems is largely tied with literacy and how it interacts with the player's imagination -- something largely in decline, at least in the US.
Sadly, you're not wrong... It might not get off the ground, but if we're able to, we'll give it a shot.
Can a MUD evaluate LLMs? A $99 proof of concept
11–20 of 88 posts
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#12[1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#13A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#14A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…
> even what can be done. I think this could be solved with bridge bots to make your MUD multiplatform. You could have a dedicated web app, an IRC bridge, a Discord bridge, etc. Discord-brained users can still use your MUD and contribute to network effects, and you can provide an off-ramp ("I just want to play VarelionScape, maybe I'll just open the dedicated website on my phone instead of going on Discord and digging…
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#15A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…
I suspect MUDs may have a bit of a resurgence soon, I've seen more and more interest in them as of late simply because they seem like an excellent pairing with LLMs which easily consume text. Though I think for your needs someone would need to work on the barrier to entry to getting on a MUD as well as the modern UI/UX expected today.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#16Speaking of MUD, has anyone tried attaching an AI to the Hitchhikers MUD[1]? [1]: https://www.bbc.co.uk/programmes/articles/1g84m0sXpnNCv84GpN...
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#17A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…
With modern technology, training in both games is very automatable. But developing and optimizing your training routines is (in my opinion) incredibly fun. Actual roleplaying is less common it was in the 90s, but there are still nightly events where dozens of players will get together to socialize and explore the worlds in-character.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#18A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#19I think you have the correct idea.
If you ask an LLM to solve a multiplication problem using reasoning without code tools, depending on the model, it will get 15 digit (15D) * 15 digit multiplication correct (123456789012345 x 998765432109876). It will take between 4000 and 8000 tokens if Sonnet. Eventually with enough digits, it will start only solving the problem 80% ... then 70% of the time until it has so many digits it will never solve. There will be a certain number of digits where it will not converge on a solution nor will it stop working thinking it can solve it. That is very, very expensive. It takes a lot of runs to determine the probability it will solve it.
What you can do now, this is likely the most important thing, is change the prompt and evaluate how many tokens and at what speed it takes to accomplish the task. Sure the measurements of each LLM are important! Nonetheless, if you can say to a company that you have a technique to tune prompts and evaluate them so instead of spending $1 X 100,000 times a day, they can instead spend $0.90 X 100,000 times a day, you will make a ton of money.
Re: Can a MUD evaluate LLMs? A $99 proof of concept
#20A little irrelevant, but I need to vent about something MUD adjacent. I really wish MUDs were still a thing. The text-based roleplaying communities were largely swallowed by Discord, where as before they existed scattered over proboards, jcink, and a few others. Discord, being the imperfect text platform, limits the text and leaves most interaction limited by awkward formatting. To boot, even when a better ecosystem…