Next up: A smart glasses companion that follows you around in the game known as life. Ive seen something like this in sci fi films.
Hopefully it’s a copper arresting you for walking around pointing cameras at people’s faces without permission.
I built a low-latency AI companion that plays Skyrim with me
71–80 of 86 posts
Re: I built a low-latency AI companion that plays Skyrim with me
#72Video game seem like the perfect fit for llm use and I don't understand why it hasn't happened already.
Why would anyone want to have an LLM generate content in a game when people can barely stomach it generating awful pictures and generic prose?
Who would want to have a non-deterministic, unreliable, out-of-your-control companion in a single player game?
What kind of game designer would find it acceptable to have no control over large swathes of their own game? Who would like to play a game that no one really created?
Games are where art meets engineering to create entertainment and awe. LLMs don’t fit.
Re: I built a low-latency AI companion that plays Skyrim with me
#73Earlier quoted context omitted.
I get it is fun to play for an hour or so then you inexorably encounter behaviors that are too complex to handle, e.g. you try to sneak in on an enemy but get spotted because the dog started talking or barking randomly, then suddenly wonder how much "it" actually gets it. You then start with less and less complex plans... until you realize that direct commands or shortcuts are more reliable. I do think it is an inter…
Kinda in the same vein, been reading/listening to Dungeon Crawler Carl lately. One of the main characters is a tortoise shell cat that gains sentience. One of the more fun aspects in the early books is the main character Carl dealing with that cat being loud, not aware of her surroundings, and making it difficult to sneak around an RPG style dungeon. Also the author really gets the "tortitude" right.
Princess Donut has insane CHA, sneaking is not the move. Just have her run in and charm everybody.
Re: I built a low-latency AI companion that plays Skyrim with me
#74Earlier quoted context omitted.
At that point it's deterministic and doesn't need the LLM. Once a plan has been decomposed and evaluated, it becomes a sequence of actions. Actions can either wait on a trigger or flow from one to another. For the arrow example: wait here -> bind: arrow event -> pick_up $target -> goto player -> drop $target. A plan can bind to multiple steps and actions. eg you can tell the dog, "when I say apple you say banana" thi…
I think what's still a bit unclear is how the action space gets linked to a plan. Did you create your own action prototypes, or are you using ones exposed via an existing Skyrim world state interface layer? How often are you running the world JSON through the planner? When do you give up on fuzzy matching if an action doesn't fit? Do you use any vision tools? such as the plugin from https://github.com/MinLL/SkyrimNet…
[deleted]
Re: I built a low-latency AI companion that plays Skyrim with me
#75This is the sort of thing that the GPT-Live model would excel at when it launches. I'm not sure you'd need the ALE model at all if they launch Live in the API with acceptable pricing as it solves the latency issues with voice entirely. It's an impressive piece of engineering that hasn't been discussed much yet, I guess because it's only a ChatGPT feature and not accessible to developers. The model can dispatch tools…
This is part of the reason I wanted to announce it even before I feel ready to ship... The frontier models have such crazy velocity (admirable) and moving so fast that I didn't want them to make an announcement and suddenly trivialize all the work I 've put into this, overnight. The fun technical challenges (that can also act as any sort of weak moat) are being taken away one by one, on an almost weekly cadence now!…
Indeed, The Bitter Lesson comes at us quicker and quicker now
Re: I built a low-latency AI companion that plays Skyrim with me
#76Earlier quoted context omitted.
Yeah, so the dog is derpy today, but the possibilities in the future are going to be awesome.
Indeed. Can’t wait until they sell me a subscription to a game that I will just look at while an AI plays and has fun for me. Whee! Awesome times ahead indeed. Or maybe it won’t play the game for me but will do whatever it wants, mess up about my gameplay because it inferred it should do something I did not order it to do. Either way, awesome! Lovely gaming experience. I remember being a kid and thinking “wouldn’t it…
you lack imagination. AI will not play the game for you.
It will construct the game around you as you play. It will customize the game according to you and the situation dynamically.
The first step is AI npcs and AI generated quest lines.
The next step is the entire world, the entire story, the entire game will be dynamically constructed as soon as you start it up.
Re: I built a low-latency AI companion that plays Skyrim with me
#77I must say absolutely hilarious video. The "persona" of the dog is great. As someone who is generally pretty "keep your AI out of my art" this looks very fun to play. I can imagine this being cleverly integrated as a primary feature of a game (this must already be in the works). Ideally as a small model able to run locally alongside the game. The moment where the dog is going on about "something foul in the air" as t…
I've noticed that the main (socially acceptable) use for AI in entertainment seems to be laughing at how bad it is. When it fucks up something obvious that any human would know not to do. When it does something so insane and out of left field that it takes you by surprise and makes you laugh. This seems to fall under that category.
Re: I built a low-latency AI companion that plays Skyrim with me
#78I must say absolutely hilarious video. The "persona" of the dog is great. As someone who is generally pretty "keep your AI out of my art" this looks very fun to play. I can imagine this being cleverly integrated as a primary feature of a game (this must already be in the works). Ideally as a small model able to run locally alongside the game. The moment where the dog is going on about "something foul in the air" as t…
I've noticed that the main (socially acceptable) use for AI in entertainment seems to be laughing at how bad it is. When it fucks up something obvious that any human would know not to do. When it does something so insane and out of left field that it takes you by surprise and makes you laugh. This seems to fall under that category.
I've noticed this too as a DougDoug fan. He seemed to be genuinely confused by general strong opposition to AI from his audience despite a lot of his content, including some of the most well-received videos, being AI-focused. I think the secret is exactly what you describe - his AI characters tend to be extremely deranged and incompetent which is what makes them hilarious.
Re: I built a low-latency AI companion that plays Skyrim with me
#79The voice of the companion is awesome.
I'm surprised how good the talk is (i.e. the voice recognition). My disabled brother is struggling with good native polish voice to text btw. If you people can recommend anything good or local
Re: I built a low-latency AI companion that plays Skyrim with me
#80This is hilariously delightful, thank you for this during my morning coffee.