Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

Post reply on HN
Print view View original

81–90 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#81

Post by koolala »

avaer wrote: post #63
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Soft disagree; if you wanted imagination you don't need to make a video model. You probably don't need to decode the latents at all. That seems pretty far from information-theoretic optimality, the kind that you want in a good+fast AI model making decisions. The whole reason for LLMs inferencing human-processable text, and "world models" inferencing human-interactive video, is precisely so that humans can connect in…

What model do you need then? If you want 3D real-time understanding of how realities work? Are you focusing on "imagination" in a different abstract way?

Re: Project Genie: Experimenting with infinite, interactive worlds

#82

Post by avaer »

analog8374 wrote: post #78

If creating an infinite world is so trivially easy (relatively speaking) then occam suggests that this world is generated.

I don't know if that's the simplest explanation, considering how insanely complex the generation is (these world models might literally be the most complex things to ever be created).

But I do think it's a partial existence proof.

Re: Project Genie: Experimenting with infinite, interactive worlds

#83

Post by jimmar »

phailhaus wrote: post #27

I have no idea why Google is wasting their time with this. Trying to hallucinate an entire world is a dead-end. There will never be enough predictability in the output for it to be cohesive in any meaningful way, by design. Why are they not training models to help write games instead? You wouldn't have to worry about permanence and consistency at all, since they would be enforced by the code, like all games today. Lo…

As a kid in the early 1980s, I spent a lot of time experimenting with computers by playing basic games and drawing with crude applications. And it was fun. I would have loved to have something like Google's Genie to play with. Even if it never evolved, the product in the demos looks good enough for people to get value from.

Re: Project Genie: Experimenting with infinite, interactive worlds

#84

Post by thegabriele »

avaer wrote: post #63
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Soft disagree; if you wanted imagination you don't need to make a video model. You probably don't need to decode the latents at all. That seems pretty far from information-theoretic optimality, the kind that you want in a good+fast AI model making decisions. The whole reason for LLMs inferencing human-processable text, and "world models" inferencing human-interactive video, is precisely so that humans can connect in…

Sure, but at some point you want humans in the loop i guess?

Re: Project Genie: Experimenting with infinite, interactive worlds

#85

Post by thegabriele »

avaer wrote: post #63
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Soft disagree; if you wanted imagination you don't need to make a video model. You probably don't need to decode the latents at all. That seems pretty far from information-theoretic optimality, the kind that you want in a good+fast AI model making decisions. The whole reason for LLMs inferencing human-processable text, and "world models" inferencing human-interactive video, is precisely so that humans can connect in…

Sure, but at some point you want humans in the loop i guess?

Re: Project Genie: Experimenting with infinite, interactive worlds

#86

Post by avaer »

MillionOClock wrote: post #57
krunck wrote: post #46

The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.

I love AI but I also hope it will paradoxically make people realize the value of real life experiences and human relationships.

Or maybe it will just make people realize the value of fake life experiences and human relationships.

Re: Project Genie: Experimenting with infinite, interactive worlds

#88

Post by echelon »

in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Whoa, whoa, whoa. That's just one angle. Please don't bin that as the only use case for "world models"!

First of all, there are a variety of different types of world models. Simulation, video, static asset, etc. It's a loaded term, just as the use cases are widespread.

There are world models you can play in your browser inferred entirely by your CPU:

https://madebyoll.in/posts/game_emulation_via_dnn/ (my favorite, from 2022!)

https://madebyoll.in/posts/world_emulation_via_dnn/ (updated, in 3D)

There are static asset generating world models, like WorldLabs' Marble. These are useful for video games, previz, and filmmaking.

https://marble.worldlabs.ai/

I wrote open source software to leverage marble for filmmaking (I'm a filmmaker, and this tech is extremely useful for scene consistency):

https://www.youtube.com/watch?v=wJCJYdGdpHg

https://github.com/storytold/artcraft

There are playable video-oriented models, many of which are open source and will run on your 3080 and above:

https://diamond-wm.github.io/

https://github.com/Robbyant/lingbot-world

There are things termed "world models" that really shouldn't be:

https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0

There are robotics training oriented world models:

https://github.com/leggedrobotics/robotic_world_model

Genie is not strictly robotics-oriented.

Re: Project Genie: Experimenting with infinite, interactive worlds

#89

Post by ACCount37 »

avaer wrote: post #63
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Soft disagree; if you wanted imagination you don't need to make a video model. You probably don't need to decode the latents at all. That seems pretty far from information-theoretic optimality, the kind that you want in a good+fast AI model making decisions. The whole reason for LLMs inferencing human-processable text, and "world models" inferencing human-interactive video, is precisely so that humans can connect in…

If you train a video model, you by necessity train a world model for 3D worlds. Which can then be reused in robotics, potentially.

I do wonder if I can frankenstein together a passable VLA using pretrained LTX-2 as a base.

Re: Project Genie: Experimenting with infinite, interactive worlds

#90

Post by adventured »

krunck wrote: post #46

The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.

Most people don't have access to anything particularly nice in the real world.

It's reality privilege. Most of humanity will yearn for the worlds that AI will cook up for them, customized to their whims.

Post reply on HN