Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

kywch.github.io

Post reply on HN
Print view View original

21–30 of 35 posts

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#21

Post by kgwxd »

yunwal wrote: post #17
kgwxd wrote: post #13

Great, add "curriculum" to the list of words that will spark my interest in human learning, only for it to be about garbage AI. I want HN with a hard rule against AI posts.

Are we really dismissing the entire field of AI just because LLMs are overhyped?

Believe it or not, you can visit more than 1 website. How about a guideline to put (AI) like we do with (video). I'm just sick of having to click to figure out if it's about humans or computers. They've hijacked every single word related to the most fascinating thing in the entire universe just to generate ad revenue and VC funding.

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#22

Post by pessimizer »

kgwxd wrote: post #21
yunwal wrote: post #17

Earlier quoted context omitted.

Are we really dismissing the entire field of AI just because LLMs are overhyped?

Believe it or not, you can visit more than 1 website. How about a guideline to put (AI) like we do with (video). I'm just sick of having to click to figure out if it's about humans or computers. They've hijacked every single word related to the most fascinating thing in the entire universe just to generate ad revenue and VC funding.

The famous Hacker News website is about computers. It is also about ad revenue and VC funding. It was originally named Startup News, and its patron and author is the multibillionaire founder of a well-known "startup accelerator" called "Y Combinator."

> Believe it or not, you can visit more than 1 website.

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#25

Post by kywch »

infinitepro wrote: post #23

Unless I am mistaken, this would be the first heuristic-free model trained to play tetris, which is pretty incredible, since mastering tetris from just raw game state has never been close to solved, till now(?)

Pufferlib already had a pretty good model before: https://puffer.ai/ocean.html?env=tetris

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#27

Post by themafia »

yunwal wrote: post #17
kgwxd wrote: post #13

Great, add "curriculum" to the list of words that will spark my interest in human learning, only for it to be about garbage AI. I want HN with a hard rule against AI posts.

Are we really dismissing the entire field of AI just because LLMs are overhyped?

LLMs show the problems of energy economy in this form of computing. It costs way too much in resources and power for minimal and generally worthless results. 2048 is a game with a several known algorithm for winning. Tetris is an obscenely simple game that unassisted humans could reliably take to the kill screen 20 years ago.

Does any of this used energy benefit any other problem?

Also using "Superhuman" in the title is absurd given this paltry outcome.

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#28

Post by ACCount37 »

pedrozieg wrote: post #7

What I like about this writeup is that it quietly demolishes the idea that you need DeepMind-scale resources to get “superhuman” RL. The headline result is less about 2048 and Tetris and more about treating the data pipeline as the main product: careful observation design, reward shaping, and then a curriculum that drops the agent straight into high-value endgame states so it ever sees them in the first place. Once y…

You never needed DeepMind scale resources to get superhuman performance on a small subset of narrow tasks. Deep Blue scale resources are often enough.

The interesting tasks, however, tend to take a lot more effort.

Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris

#29

Post by kywch »

gyrovagueGeist wrote: post #19

I've always found curriculum learning incredibly hard to tune and calibrate reliably (even more so than many other RL approaches!). Reward scales and horizon lengths may vary across tasks with different difficulty, effectively exploring policy space (keeping multimodal strategy distributions for exploration before overfitting on small problems), and catastrophic forgetting when mixing curriculum levels or when introd…

I think Go-Explore (https://arxiv.org/abs/1901.10995) is promising. It'll provide automatic scaffolding and prevent catastrophic forgetting.

If one can frame the problem into a competition, then self-play has been shown to work repeatedly.

Post reply on HN