Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: GPT crushes my high score in 2048.io

github.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Show HN: GPT crushes my high score in 2048.io

#21

Post by jychang »

sdflhasjd wrote: post #7

I can't think of a particularly polite way of putting this, so I apologise, but did you take some liberties with that title, or are you just not very good at 2048? Random inputs may get you a higher score than that. I looked at the code to see the prompt, and I think it's a very limited way of having GPT play, the context has no board history or information on how and where the game inserts piece, the AI won't be abl…

OP might be my little brother. Years back, I destroyed him in 2048, and then he called me a virgin. Then he wrecked me at a round of chess.

(I was engaged then, currently married now)

Ahh, good times.

Re: Show HN: GPT crushes my high score in 2048.io

#25

Post by lordnacho »

This is a pretty interesting example, as I think most of you will find that after playing 2048 for a bit, you discover a way to beat the game every time. I'm not sure I could code it up, but the heuristic is basically to keep the big numbers squashed against the same edge (eg the top) the whole game while using left-right-up movements to squash the smaller numbers into each other as opportunities arise.

I don't think GPT could figure this out. My impression of it lately is that it's a sort of very advanced cargo cultist, maybe with a bit of superficial intelligence confined to the linguistic sphere. Asking it for a history essay gives you a grammatically perfect melange of likely terms that will do just fine for high school but possibly not for graduate level studies.

I've never seen it do anything where I thought it had a parsimonious internal model of the problem. For instance I had it tell me about the quadratic equation, and the explanation was fine. When it came to plugging in numbers, it utterly failed, though the presentation was as if it understood it. If it had just a simple calculator inside it, this wouldn't be a problem.

This game is also pretty simple, and for the same reason I don't think it can actually do it.

Re: Show HN: GPT crushes my high score in 2048.io

#26

Post by pmoriarty »

kazinator wrote: post #3

GPT4 just told me that a billion golf balls take up 40 billion cc, and since packing efficiency of spheres is as high as 74%, they take up 26 billion cc. No, you have to divide by 0.74, not multiply. In shit that GPT4 is not trained on (like code code code and more code), it can get really goofy. Earlier in the same chat about golf balls, it claimed that if brain cells were the size of golf balls (an imaginary thread…

It would be interesting to know how Anthropic's Claude+[1] (or at least plain old Claude[2]) would fare on these tests.

[1] - https://poe.com/Claude%2B

[2] - https://poe.com/Claude-instant

Re: Show HN: GPT crushes my high score in 2048.io

#27

Post by Kaotique »

Kaotique wrote: post #19
jxf wrote: post #17

Earlier quoted context omitted.

There shouldn't need to be any board history, just like you don't need to know the board history to know the next move in checkers.

2048 spawns new numbers randomly on the board. A human player uses that information to determine the best move.

Besides that you need to predict where the next number can and should spawn. If you don't figure out the strategy for that you will never get a decent score.

Re: Show HN: GPT crushes my high score in 2048.io

#28

Post by morelisp »

Kaotique wrote: post #19
jxf wrote: post #17

Earlier quoted context omitted.

There shouldn't need to be any board history, just like you don't need to know the board history to know the next move in checkers.

2048 spawns new numbers randomly on the board. A human player uses that information to determine the best move.

Board history does not matter.

    Grid.prototype.randomAvailableCell = function () {
      var cells = this.availableCells();

      if (cells.length) {
        return cells[Math.floor(Math.random() * cells.length)];
      }
    };

Re: Show HN: GPT crushes my high score in 2048.io

#29

Post by Kaotique »

mkl95 wrote: post #16

It looks like the wrong AI for the problem. RL should be more successful.

AlphaStar or AlphaGo were trained on thousands of games and played hundreds of thousands of games to learn what works and what doesn't. This model is told the rules of the game and knows how to use the 4 basic inputs. I don't think it has any notion of strategy, good or bad moves.

But maybe with a feedback loop it will improve.

Pretty cool example though to see its limitations.

Re: Show HN: GPT crushes my high score in 2048.io

#30

Post by gumballindie »

ggerganov wrote: post #13
have_faith wrote: post #9

Earlier quoted context omitted.

What would be a good benchmark score for an AI? I just got 4500~ without any strategy.

Just make the same Javscript that provides random moves, make it play 100 games and take the average score. Then do the same using GPT and compare the scores. Anything else is just cherry-picking

Yeah but that doesn't drive clicks.
Post reply on HN