Ha, interesting. I wasn't aware of Sutton's blog post, but if I might make a shameless plug, we demonstrated [1] exactly this problem (see section 4.4.3), and how multi-step world models (using diffusion models as the substrate) could be one potential answer. Since then, I have come to like temporally-abstract models more and more. Rolling out in time -- either step-by-step or many steps at once -- suffers from the t…
What do you mean by tyranny of the specific?
Imagine I want to attend a conference in a different country. Google maps might give turn by turn navigation but that is an overwhelming and largely irrelevant mess of details for most planning purposes. Eg: all I might want to know is the different flight legs and the fact that the journey takes 15-18 hours, and not all the turns and traffic lights to get from home to the airport.
I want a zoomed out picture, and to be able to fill in detail hierarchically, on demand. Instead, one-step models give you the full high-res local structure of the graph that would have to search through (with too many states and edges).