World Emulation via Neural Network
madebyoll.in
World Emulation via Neural Network
1–10 of 48 posts
Re: World Emulation via Neural Network
#2Is this a solo/personal project? If it is is indeed very cool.
Is OP the blog’s author? Because in the post the author said that the purpose of the project is to show why NN are truly special and I wanted a more articulate view of why he/she thinks that? Good work anyway!
Re: World Emulation via Neural Network
#3Is this a solo/personal project? If it is is indeed very cool. Is OP the blog’s author? Because in the post the author said that the purpose of the project is to show why NN are truly special and I wanted a more articulate view of why he/she thinks that? Good work anyway!
author is: https://x.com/madebyollin
Re: World Emulation via Neural Network
#4Appreciate this article that shows some failures on the way to a great result. Too many times, people only show how the polished end-result: look, I trained this AI and it produces these great results. The world dissolving was very interesting to see, even if I'm not sure I understand how it got fixed.
Re: World Emulation via Neural Network
#5This is great but I think I'll stick to mushrooms.
Re: World Emulation via Neural Network
#6Is this a solo/personal project? If it is is indeed very cool. Is OP the blog’s author? Because in the post the author said that the purpose of the project is to show why NN are truly special and I wanted a more articulate view of why he/she thinks that? Good work anyway!
Yes! This was a solo project done in my free time :) to learn about WMs and get more practice training GANs.
The special aspect of NNs (in the context of simulating worlds) is that NNs can mimic entire worlds from videos alone, without access to the source code (in the case of pokemon) or even without the source code having existed (as is the case for the real-world forest trail mimicked in this post). They mimic the entire interactive behavior of the world, not just the geometry (note e.g. the not-programmed-in autoexposure that appears when you look at the sky).
Although the neural world in the post is a toy project, and quite far from generating photorealistic frames with "trees that bend in the wind, lilypads that bob in the rain, birds that sing to each other", I think getting better results is mostly a matter of scale. See e.g. the GAIA-2 results (https://wayve.ai/wp-content/uploads/2025/03/generalisation_0..., https://wayve.ai/wp-content/uploads/2025/03/unsafe_ego_01_le...) for an example of what WMs can do without the realtime-rendering-in-a-browser constraints :)
Re: World Emulation via Neural Network
#7Appreciate this article that shows some failures on the way to a great result. Too many times, people only show how the polished end-result: look, I trained this AI and it produces these great results. The world dissolving was very interesting to see, even if I'm not sure I understand how it got fixed.
Thanks! My favorite failure mode (not mentioned in the post - I think it was during the first round of upgrades?) was a "dry" form of soupification where the texture detail didn't fully disappear https://imgur.com/c7gVRG0
Re: World Emulation via Neural Network
#8Really cool. How much compute did you require to successfully train these models? Is it in the ballpark of something you could do with a single gaming GPU? Or did you spin up something fancier?
edit: I see now that you mention a pricepoint of 100 GPU-hours/roughly 100$. My mistake.
Re: World Emulation via Neural Network
#9Is this a solo/personal project? If it is is indeed very cool. Is OP the blog’s author? Because in the post the author said that the purpose of the project is to show why NN are truly special and I wanted a more articulate view of why he/she thinks that? Good work anyway!
Yes! This was a solo project done in my free time :) to learn about WMs and get more practice training GANs. The special aspect of NNs (in the context of simulating worlds) is that NNs can mimic entire worlds from videos alone, without access to the source code (in the case of pokemon) or even without the source code having existed (as is the case for the real-world forest trail mimicked in this post). They mimic the…
You mentioned it took 100 gpu hours, what gpu did you train on?
Re: World Emulation via Neural Network
#10I want to see a spiritual successor to LSD: Dream Emulator based on this.