Is there value in using deep RL for problems that seem more suited to planning-based approaches?
Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris
31–35 of 35 posts
Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris
#32Earlier quoted context omitted.
perhaps I'm missing something. Why not start the learning at a later state?
If the goal is to achieve end-to-end learning that would be cheating. If you sat down to solve a problem you’ve never seen before you wouldn’t even know what a valid “later state” looking like.
Why is it cheating? We literally teach sports this way? Often times you teach sports by learning in scaled down scenarios. I see no reason this should be different.
Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris
#33I wonder if he tried NNUE
NNUE is for deep searches, as far as I understand this just says what move to do based on the state?
Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris
#34Earlier quoted context omitted.
If the goal is to achieve end-to-end learning that would be cheating. If you sat down to solve a problem you’ve never seen before you wouldn’t even know what a valid “later state” looking like.
Why is it cheating? We literally teach sports this way? Often times you teach sports by learning in scaled down scenarios. I see no reason this should be different.
If the goal is to learn how to solve a Rubik's Cube when you've never seen a Rubik's Cube before, you have no idea what "halfway solved" even looks like.
This is precisely how RL worked for learning Atari games: you don't start with the game halfway solved and then claim the AI solved the end-to-end problem on its own.
The goal in these scenarios is for the machine to solve the problem with no prior information.
Re: Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris
#35Earlier quoted context omitted.
Why is it cheating? We literally teach sports this way? Often times you teach sports by learning in scaled down scenarios. I see no reason this should be different.
If the goal is to learn how to solve a Rubik's Cube when you've never seen a Rubik's Cube before, you have no idea what "halfway solved" even looks like. This is precisely how RL worked for learning Atari games: you don't start with the game halfway solved and then claim the AI solved the end-to-end problem on its own. The goal in these scenarios is for the machine to solve the problem with no prior information.
This isn't accurate, though? Halfway solved, for most teachings, is to have the first layer solved.
Indeed, this is a key to teaching people to know how to advance. Do not focus on a side, but learn to advance a layer.