Why are large language models so terrible at video games?
spectrum.ieee.org
Why are large language models so terrible at video games?
1–10 of 64 posts
Re: Why are large language models so terrible at video games?
#2> This brings us to what seems like a contradiction. LLMs are bad at playing games. Yet at the same time, they’re improving rapidly at coding, a skill set that can be used to create a game. How do these facts fit together?
> Togelius: It’s super weird.
...No, it's really not.
They're language models. Code is a language. "Playing a game well" is not. One can, hypothetically, encode game inputs in such a way that it seems kinda-sorta like a language, but it has none of the same kinds of structures that languages—both human and programming—do.
The only way one can think this is strange is if one thinks of LLMs' ability to code rudimentary games as being due to a deeper understanding of games, rather than due to game code being well-represented in their training data.
Re: Why are large language models so terrible at video games?
#3Video games are made to entertain humans, so does it really matter whether LLMs are good at playing them?
Re: Why are large language models so terrible at video games?
#4cough JEPA cough
Re: Why are large language models so terrible at video games?
#5Video games are made to entertain humans, so does it really matter whether LLMs are good at playing them?
[deleted]
Re: Why are large language models so terrible at video games?
#6Its almost like the Large Language Model has trouble with things that arent Language, such as realtime controller input and video output from a game
Re: Why are large language models so terrible at video games?
#7Video games are made to entertain humans, so does it really matter whether LLMs are good at playing them?
It matters a lot because it's a real solution for external bots that plays more "fairly" especially in older games. It also allows to test games autonomously, which is huge if we are talking about automated programming.
Imagine if you can bring those AI players to CS 1.6.
Re: Why are large language models so terrible at video games?
#8Video games are made to entertain humans, so does it really matter whether LLMs are good at playing them?
It matters a lot because it's a real solution for external bots that plays more "fairly" especially in older games. It also allows to test games autonomously, which is huge if we are talking about automated programming. Imagine if you can bring those AI players to CS 1.6.
LLMs are the wrong tool for video games. There have been plenty of successful non-LLM AIs that have been trained with reinforcement learning to play games.
If you want to implement actual bots inside the game, then you want to use explicit logic instead of inferred logic. It's much more efficient and easier to debug.
If you want to create Bots for an existing game, which doesn't have its own pre-programmed bots, then you should look at other types of AI. See https://www.geeksforgeeks.org/deep-learning/reinforcement-le...
Re: Why are large language models so terrible at video games?
#9I actually really miss all the research being done on having (reinforcement learning) AIs beat Atari games and the like. Or the one that stopped at a TV playing random images instead of continuing through the level. Has there been any progress in that field? It seems like LLMs came around and all the projects stopped completely.
Re: Why are large language models so terrible at video games?
#10There was good progress in training neural networks to play video games.
Unfortunately it doesn't seem to fit in some people's context because it was a few years ago.
Kind reminder: there is "AI" beyond LLMs.