Live data from Hacker News

Why are large language models so terrible at video games?

spectrum.ieee.org

21–30 of 64 posts

Re: Why are large language models so terrible at video games?

#21

I wonder if you paired a few different types of AI together, an LLM agent might be good at strategizing -. E.g. building a strategy on how to handle a scenario. But, it would need to know the entire game manual basically. Then it would pass the stratrgy to a better AI in some way. But it might not be needed if the better gaming AI can just do that part too already. I admit I know nothing about this though.

GOAP is a better tool.

Re: Why are large language models so terrible at video games?

#24
post #10

There was good progress in training neural networks to play video games. Unfortunately it doesn't seem to fit in some people's context because it was a few years ago. Kind reminder: there is "AI" beyond LLMs.

OpenAI's Dota 2 adventures were super hype back in the days.

Re: Why are large language models so terrible at video games?

#25
post #16
post #2

> This brings us to what seems like a contradiction. LLMs are bad at playing games. Yet at the same time, they’re improving rapidly at coding, a skill set that can be used to create a game. How do these facts fit together? > Togelius: It’s super weird. ...No, it's really not. They're language models. Code is a language. "Playing a game well" is not. One can, hypothetically, encode game inputs in such a way that it se…

Yet LLMs can play chess and have a "mental" representation of the chessboard. If LLMs get better but do not progress at playing games when not specifically trained on it it seems to point to a generalisation failure, a limitation that would prevent LLMs to ever achieve AGI, I do not know if that is weird but it seems that for now nobody really knows if they can achieve AGI or not. Perhaps some emergent behavior will…

Chess games are in their training set, other games are not.

Re: Why are large language models so terrible at video games?

#26
As others have hinted at LLMs aren't really made in a way that makes them likely to play video games (CS/Halo and such) well. I wonder how they'd fare "against" text based adventures like Zork (which they'll no doubt have ample knowledge about) and newer text based adventure games (which they'll know less about).

Re: Why are large language models so terrible at video games?

#28
post #10

There was good progress in training neural networks to play video games. Unfortunately it doesn't seem to fit in some people's context because it was a few years ago. Kind reminder: there is "AI" beyond LLMs.

OpenAI's Dota 2 adventures were super hype back in the days.

OpenAI Five doesn’t really know how to play games in general — it only knows how to play Dota.

Re: Why are large language models so terrible at video games?

#29
post #10

There was good progress in training neural networks to play video games. Unfortunately it doesn't seem to fit in some people's context because it was a few years ago. Kind reminder: there is "AI" beyond LLMs.

Several years ago I built a simple snake game and wrote a DQN from scratch to learn how to play it.

I was really proud of it at the time because I had to do a decent amount of reading and research since I wrote all of the NN code from scratch and wanted to add some more advanced algorithm optimisations which I hadn't done in previous projects.

I suspect a coding agent could spit the entire project out in 20 minutes now, but it was very cool at the time to build a game then watch my computer learn how to play it in real time.

Re: Why are large language models so terrible at video games?

#30
I guess the author’s point is that LLMs can’t really learn in real time yet, whereas playing games is basically all about real-time learning. So an LLM can be very good at writing code, but still be terrible at actually playing games.

Personally, I think this is a really hard problem, and it may turn out to be one of the first big walls we hit on the road to AGI.

Post reply on HN