Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

Post reply on HN
Print view View original

111–120 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#111

Post by 2bitencryption »

slashdave wrote: post #106
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

This is a video model, not a world model. Start learning on this, and cascading errors will inevitably creep into all downstream products. You cannot invent data.

Given that the video is fully interactive and lets you move around (in a “world” if you will) I don’t think it’s a stretch to call it a world model. It must have at least some notion of physics, cause and effect, etc etc in order to achieve what it does.

Re: Project Genie: Experimenting with infinite, interactive worlds

#112

Post by jsheard »

falcor84 wrote: post #67
jsheard wrote: post #15

Earlier quoted context omitted.

Isn't this still essentially "vibe simulation" inferred from videos? Surface-level visual realism is one thing, but expecting it to figure out the exact physical mechanics of sailing just by watching boats, and usefully abstract that into a gamified form, is another thing entirely.

Why wouldn't it just hook it into something like physx?

Google has made it clear that Genie doesn't maintain an explicit 3D scene representation, so I don't think hooking in "assists" like that is on the table. Even if it were, the AI layer would still have to infer things like object weight, density, friction and linkages correctly. Garbage in, garbage out.

Re: Project Genie: Experimenting with infinite, interactive worlds

#114

Post by slashdave »

sy26 wrote: post #10

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

Failures are not publicly reported, in general. Do you we know what they have invested in?

Re: Project Genie: Experimenting with infinite, interactive worlds

#115

Post by YetAnotherNick »

qwertyi0k wrote: post #62
observationist wrote: post #26

Earlier quoted context omitted.

LeCun wasn't producing results. He was obstinate and insistent on his own theories and ideas which weren't, and possibly aren't, going anywhere. He refused to engage with LLMs and compete in the market that exists, and spent all his effort and energy on unproven ideas and research, which split the company's mission and competitiveness. They lost their place as one of the top 4 AI companies, and are now a full generat…

To be fair, this was his job description: Fundamental AI Research (FAIR) lab. Not AI products division. You can't expect marketable products from a fundamental AI research lab.

It's "Facebook Artificial Intelligence Research", not fundamental. So basically involves both fundamental and applied research.

[1]: https://engineering.fb.com/category/ai-research/

Re: Project Genie: Experimenting with infinite, interactive worlds

#116

Post by hagbarth »

0xcb0 wrote: post #7

I keep on repeating myself, but it feels like I'm living in the future. Can't wait to hook this up to my old Oculus glasses and let Genie create a fully realistic sailing simulator for me, where I can train sailing with realistic conditions. On boats I'd love to sail. If making games out of these simulations work, it't be the end for a lot of big studios, and might be the renaissance for small to one person game stud…

> and might be the renaissance for small to one person game studios.

Indie games are already bigger than ever as far as I know.

Re: Project Genie: Experimenting with infinite, interactive worlds

#117

Post by in-silico »

echelon wrote: post #101
in-silico wrote: post #98

Earlier quoted context omitted.

The entertainment industry, as big as it is, just doesn't have as much profit potential as robots and AI agents that can replace human labor. Just look at how Nvidia has pivoted from gaming and rendering to AI. The other examples you've given are neat, but for players like Google they are mostly an afterthought.

Robotics: $88B TAM Gaming: $350B TAM All media and entertainment: $3T TAM Manufacturing: $5T TAM Roughly the same story. This tech is going to revolutionize "films" and gaming. The entire entertainment industry is going to transform around it. When people aren't buying physical things, they're distracting themselves with media. Humans spend more time and money on that than anything else. Machines or otherwise. AI imp…

The current robotics industry is $88B. You have to take into account the potential future industry of general purpose robots that replace a big chunk of blue-collar work.

Robots is also just one example. A hypothetically powerful AI agent (which might also use a world model) that controls a mouse and keyboard could replace a big chunk of white-collar work too.

Those are worth 10's of trillions of dollars. You can argue about whether they are actually possible, but the people backing this tech think they are.

Re: Project Genie: Experimenting with infinite, interactive worlds

#118

Post by hatmanstack »

reactordev wrote: post #102
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Still cool though…

this.

Re: Project Genie: Experimenting with infinite, interactive worlds

#119

Post by echelon »

avaer wrote: post #79
dyauspitr wrote: post #75

Earlier quoted context omitted.

That’s part of it but if you could actually pull out 3D models from these worlds, it would massively speed up game development.

You already can, check out Marble/World Labs, Meshy, and others. It's not really as much of a boon as you'd think though, since throwing together a 3D model is not the bottleneck to making a sellable video game. You've had model marketplaces for a long time now.

> It's not really as much of a boon as you'd think though

It is for filmmaking! They're perfect for constructing consistent sets and blocking out how your actors and props are positioned. You can freely position the camera, control the depth of field, and then storyboard your entire scene I2V.

Example of doing this with Marble: https://www.youtube.com/watch?v=wJCJYdGdpHg

Re: Project Genie: Experimenting with infinite, interactive worlds

#120

Post by ollin »

montebicyclelo wrote: post #6

Reminds me of this [1] HN post from 9 months ago, where the author trained a neural network to do world emulation from video recordings of their local park — you can walk around in their interactive demo [2]. I don't have access to the DeepMind demo, but from the video it looks like it takes the idea up a notch. (I don't know the exact lineage of these ideas, but a general observation is that it's a shame that it's t…

Yup, similar concepts! Just at two opposite extremes of the compute/scaling spectrum.

- That forest trail world is ~5 million parameters, trained on 15 minutes of video, scoped to run on a five-year-old iPhone through a twenty-year old API (WebGL GPGPU, i.e OpenGL fragment shaders). It's the smallest '3D' world model I'm aware of.

- Genie 3 is (most likely) ~100 billion parameters trained on millions of hours of video and running across multiple TPUs. I would be shocked if it's not the largest-scale world model available to the public.

There are lots of neat intermediate-scale world models being developed as well (e.g. LingBot-World https://github.com/robbyant/lingbot-world, Waypoint 1 https://huggingface.co/blog/waypoint-1) so I expect we'll be able to play something of Genie quality locally on gaming GPUs within a year or two.

Post reply on HN