Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

tetrisbench.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#31

Post by Keyframe »

bubblesorting wrote: post #4

Very cool! I am a good Tetris player (in the top 10% of players) and wanted to give brick yeeting against an LLM a spin. Some feedback: - Knowing the scoring system is helpful when going 1v1 high score - Use a different randomization system, I kept getting starved for pieces like I. True random is fine, throwing a copy of every piece into a bag and then drawing them one by one is better (7 bag), nearly random with so…

cool to hear about more randomizers. I am not a great tetris player but I absolutely love the game. I put NES, 7-bag and my own take on randomizer (crap) in my online tetris-like experiment https://www.susmel.com/stacky/ (you can press c for more controls or h to see shortcuts)

One of my dream goals was to make a licensed low lag competitive game kind of like TGM, but I heard licensing is extremely cost-prohibitive so I kind of gave up on that goal. I remember I said to someone I was ready to pony up few tens of thousands for a license + cut, but reportedly it starts at an order of the magnitude higher.

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#32

Post by augusteo »

LLMs playing Tetris feels like testing a calculator's ability to write poetry. Interesting as a curiosity, but the results don't transfer to the tasks where these models actually excel.

Curious what the latency looks like per move. That seems like the actual bottleneck here.

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#33

Post by akomtu »

gpm wrote: post #13
akomtu wrote: post #2

It would be more interesting to make it build a chess engine and compare it against Stockfish. The chess engine should be a standalone no-dependencies C/C++ program that fits in NNN lines of code.

Comparing against stockfish isn't fair. That's comparing against enormous amounts of compute spent experimenting with strategies, training neutral nets, etc. It will lose so badly there will be no point in the comparison. Besides you could compare models (and harnesses) directly against eachother.

Stockfish is a good reference point, an objective measure of how far the LLM's advanced.

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#34

Post by dakom »

ykhli wrote: post #17

Thanks for all the questions! More details on how this works: - Each model starts with an initial optimization function for evaluating Tetris moves. - As the game progresses, the model sees the current board state and updates its algorithm—adapting its strategy based on how the game is evolving. - The model continuously refines its optimizer. It decides when it needs to re-evaluate and when it should implement the ne…

How does it deal with latency? Afaict remote LLMs need seconds to process, but Tetris can move much faster..

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#35

Post by ykhli »

indigodaddy wrote: post #24

Is there a tl;dr on why this is? Does it just make faster decisions?

my unvalidated theory is that this comes down to the coding model’s training objective: Tetris is fundamentally an optimization problem with delayed rewards. Some models seem to aggressively over-optimize toward near term wins (clearing lines quickly), which looks good early but leads to brittle states and catastrophic failures later. Others appear to learn more stable heuristics like board smoothness, height control, long-term survivability even if that sacrifices short-term score

That difference in objective bias shows up very clearly in Tetris, but is much harder to notice in typical coding benchmarks. Just a theory though based on reviewing game results and logs

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#36

Post by brookman64k »

p0w3n3d wrote: post #21

Guys, I don't know how to tell you but... Tetris can web solved without LLM...

Why run something in a few CPU cyles on a 40 year old home computer when you can do the same (but worse) on a billion-dollar GPU cluster?

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#37

Post by mikkupikku »

akomtu wrote: post #33
gpm wrote: post #13

Earlier quoted context omitted.

Comparing against stockfish isn't fair. That's comparing against enormous amounts of compute spent experimenting with strategies, training neutral nets, etc. It will lose so badly there will be no point in the comparison. Besides you could compare models (and harnesses) directly against eachother.

Stockfish is a good reference point, an objective measure of how far the LLM's advanced.

It's not. Maybe if you used old versions of stockfish that predate the neural net methods used by current versions, because otherwise you'd be comparing the hand-rolled (by an LLM) position evaluation functions against an NNUE and the results of that are a forgone conclusion; stockfish will stomp it every time.

Maybe that's the result you want for some sort of rhetorical reason, but it would nonetheless not be an informative test.

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#38

Post by storystarling »

ykhli wrote: post #20
vunderba wrote: post #12

Interesting but frustratingly vague on details. How exactly are the models playing? Is it using some kind of PGN equivalent in Tetris that represents a on-going game, passing an ASCII representation, encoding as a JSON structure, or just directly sending screenshots of the game to the various LLMs?

answered this in a comment above! It's not turn or visual layout based since LLMs are not trained that way. The representation is a JSON structure, but LLMs plug in algorithms and keeps optimizing it as the game state evolves

Curious how the token economics compare here to a standard agent loop. It seems like if you're using the LLM as a JIT to optimize the algorithm as the game evolves, the context accumulation would get expensive fast even with Flash pricing.

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#39

Post by p0w3n3d »

brookman64k wrote: post #36
p0w3n3d wrote: post #21

Guys, I don't know how to tell you but... Tetris can web solved without LLM...

Why run something in a few CPU cyles on a 40 year old home computer when you can do the same (but worse) on a billion-dollar GPU cluster?

Maybe this is about parsing the video? But still this should be done with OpenCV and then algorithmically...

A youtuber sentdex has a whole series on parsing the game's image and playing GTA https://www.youtube.com/playlist?list=PLQVvvaa0QuDeETZEOy4Vd...

Re: Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus

#40

Post by gpm »

p0w3n3d wrote: post #21

Guys, I don't know how to tell you but... Tetris can web solved without LLM...

The world record for hatetris (tetris with deterministic somewhat adversarial piece selection instead of random piece selection) is still finite. The game is not solved. https://qntm.org/records
Post reply on HN