Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

Post reply on HN
Print view View original

91–100 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#91

Post by arionmiles »

seedie wrote: post #56
phailhaus wrote: post #27

I have no idea why Google is wasting their time with this. Trying to hallucinate an entire world is a dead-end. There will never be enough predictability in the output for it to be cohesive in any meaningful way, by design. Why are they not training models to help write games instead? You wouldn't have to worry about permanence and consistency at all, since they would be enforced by the code, like all games today. Lo…

Imo they explain pretty well what they are trying to achieve with SIMA and Genie in the Google Deepmind Podcast[1]. They see it as the way to get to AGI by letting AI agents learn for themselves in simulated worlds. Kind of like how they let AlphaGo train for Go in an enormous amount of simulated games. [1] https://youtu.be/n5x6yXDj0uo

If that's the goal, the technology for how these agents "learn" would be the most interesting one, even more than the demos in the link.

LLMs can barely remember the coding style I keep asking it to stick to despite numerous prompts, stuffing that guideline into my (whatever is the newest flavour of product-specific markdown file). They keep expanding the context window to work around that problem.

If they have something for long-term learning and growth that can help AI agents, they should be leveraging it for competitive advantage.

Re: Project Genie: Experimenting with infinite, interactive worlds

#92

Post by RaftPeople »

ollin wrote: post #28

Really great to see this released! Some interesting videos from early-access users: - https://youtu.be/15KtGNgpVnE?si=rgQ0PSRniRGcvN31&t=197 walking through various cities - https://x.com/fofrAI/status/2016936855607136506 helicopter / flight sim - https://x.com/venturetwins/status/2016919922727850333 space station, https://x.com/venturetwins/status/2016920340602278368 Dunkin' Donuts - https://youtu.be/lALGud1Ynhc?si=…

I liked that first one and I hope someone creates one of going back to dinosaur age, i want to see that.

Re: Project Genie: Experimenting with infinite, interactive worlds

#93

Post by echelon »

krunck wrote: post #46

The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.

The more I see of this, the faster I want it to go!

I'm developing filmmaking tools with World Labs' Marble world model:

https://www.youtube.com/watch?v=wJCJYdGdpHg

https://github.com/storytold/artcraft

I think we'll eventually get to the point where these are real time and have consistent representations. I've been excited about world models since I saw the in-the-browser Pokemon demo:

https://madebyoll.in/posts/game_emulation_via_dnn/demo/

At some point, we'll have the creative Holodeck. If you've seen what single improv performers can do with AI, it's ridiculously cool. I can imagine watching entertainers in the future that summon and create entire worlds before us:

https://www.youtube.com/watch?v=MYH3FIFH55s

(If you haven't seen CodeMiko, she's an incredibly talented engineer and streamer. She develops mocap + AI streams.)

Re: Project Genie: Experimenting with infinite, interactive worlds

#94

Post by benlivengood »

in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Agreed; everyone complained that LLMs have no world model, so here we go. Next logical step is to backfill the weights with encoded video from the real world at some reasonable frame rate to ground the imagination and then branch the inference on possible interventions (actions) in the near future of the simulation, throw the results into a goal evaluator and then send the winning action-predictions to motors. Getting timing right will probably require a bit more work than literally gluing them together, but probably not much more.

Re: Project Genie: Experimenting with infinite, interactive worlds

#95

Post by mapmeld »

observationist wrote: post #26
sy26 wrote: post #10

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

LeCun wasn't producing results. He was obstinate and insistent on his own theories and ideas which weren't, and possibly aren't, going anywhere. He refused to engage with LLMs and compete in the market that exists, and spent all his effort and energy on unproven ideas and research, which split the company's mission and competitiveness. They lost their place as one of the top 4 AI companies, and are now a full generat…

In an industry of big bets, especially considering the company has poured resources and renamed itself to secure a place in the VR world... staking your reputation on everyone's LLMs having peaked and shifting focus to finding a new path to AI is a pretty interesting bet, no?

Re: Project Genie: Experimenting with infinite, interactive worlds

#96

Post by analog8374 »

avaer wrote: post #82
analog8374 wrote: post #78

If creating an infinite world is so trivially easy (relatively speaking) then occam suggests that this world is generated.

I don't know if that's the simplest explanation, considering how insanely complex the generation is (these world models might literally be the most complex things to ever be created). But I do think it's a partial existence proof.

One big simplifier is to only render what you're looking at. I wonder how one might demonstrate that.

Re: Project Genie: Experimenting with infinite, interactive worlds

#97

Post by echelon »

RaftPeople wrote: post #92
ollin wrote: post #28

Really great to see this released! Some interesting videos from early-access users: - https://youtu.be/15KtGNgpVnE?si=rgQ0PSRniRGcvN31&t=197 walking through various cities - https://x.com/fofrAI/status/2016936855607136506 helicopter / flight sim - https://x.com/venturetwins/status/2016919922727850333 space station, https://x.com/venturetwins/status/2016920340602278368 Dunkin' Donuts - https://youtu.be/lALGud1Ynhc?si=…

I liked that first one and I hope someone creates one of going back to dinosaur age, i want to see that.

Tim is awesome.

Ironically, he covered PixVerse's world model last week and it came close to your ask: https://youtu.be/SAjKSRRJstQ?si=dqybCnaPvMmhpOnV&t=371

(Earlier in the video it shows him live prompting.)

World models are popping up everywhere, from almost every frontier lab.

Re: Project Genie: Experimenting with infinite, interactive worlds

#98

Post by in-silico »

echelon wrote: post #88
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Whoa, whoa, whoa. That's just one angle. Please don't bin that as the only use case for "world models"! First of all, there are a variety of different types of world models. Simulation, video, static asset, etc. It's a loaded term, just as the use cases are widespread. There are world models you can play in your browser inferred entirely by your CPU: https://madebyoll.in/posts/game_emulation_via_dnn/ (my favorite, fr…

The entertainment industry, as big as it is, just doesn't have as much profit potential as robots and AI agents that can replace human labor. Just look at how Nvidia has pivoted from gaming and rendering to AI.

The other examples you've given are neat, but for players like Google they are mostly an afterthought.

Re: Project Genie: Experimenting with infinite, interactive worlds

#99

Post by Legend2440 »

nozbufferHere wrote: post #64
WarmWash wrote: post #33

The actual breakthrough with Genie is being able to turn around and look back, and seeing the same scene that was there before. A few other labs have similar world simulators, but they all struggle badly with keeping coherence of things not in view. Hence why they always walk forwards and never look around.

Still amazed it took ML people so long to realize they needed and explicit representation to cache stuff.

Genie does not use an explicit representation:

>Genie 3’s consistency is an emergent capability. Other methods such as NeRFs and Gaussian Splatting also allow consistent navigable 3D environments, but depend on the provision of an explicit 3D representation. By contrast, worlds generated by Genie 3 are far more dynamic and rich because they’re created frame by frame based on the world description and actions by the user.

Re: Project Genie: Experimenting with infinite, interactive worlds

#100

Post by autonomousErwin »

jsheard wrote: post #47
sfn42 wrote: post #45

Earlier quoted context omitted.

And what if I go somewhere then go back there a week later?

Best they can do is 60 seconds, for now at least.

Makes you wonder what the TTL caching for our universe is.
Post reply on HN