Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

Post reply on HN
Print view View original

171–180 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#171

Post by post-it »

whalee wrote: post #159
phailhaus wrote: post #27

I have no idea why Google is wasting their time with this. Trying to hallucinate an entire world is a dead-end. There will never be enough predictability in the output for it to be cohesive in any meaningful way, by design. Why are they not training models to help write games instead? You wouldn't have to worry about permanence and consistency at all, since they would be enforced by the code, like all games today. Lo…

This was a common argument against LLMs, that the space of possible next tokens is so vast that eventually a long enough sequence will necessarily decay into nonsense, or at least that compounding error will have the same effect. Problem is, that's not what we've observed to happen as these models get better. In reality there is some metaphysical coarse-grained substrate of physics/semantics/whatever[1] which these m…

And going back a little further, it was thought that backpropagation would be impractical, and trying to train neural networks was a dead end. Then people tried it and it worked just fine.

Re: Project Genie: Experimenting with infinite, interactive worlds

#172

Post by Sol- »

MillionOClock wrote: post #57
krunck wrote: post #46

The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.

I love AI but I also hope it will paradoxically make people realize the value of real life experiences and human relationships.

Dunno, I want to agree, but at the same time it's spoken like someone to whom these experiences and human relationship come easily. There are many people out there who, for some reason (anxiety, etc.), cannot easily access this part of the human condition, unfortunately.

Perhaps better to roam a virtual reality than be starved in the real world.

Re: Project Genie: Experimenting with infinite, interactive worlds

#173

Post by emmettm »

nozbufferHere wrote: post #64
WarmWash wrote: post #33

The actual breakthrough with Genie is being able to turn around and look back, and seeing the same scene that was there before. A few other labs have similar world simulators, but they all struggle badly with keeping coherence of things not in view. Hence why they always walk forwards and never look around.

Still amazed it took ML people so long to realize they needed and explicit representation to cache stuff.

The representation is learned. Also, see Sutter's "Bitter Lesson" essay

Re: Project Genie: Experimenting with infinite, interactive worlds

#174

Post by trenning »

moomoo11 wrote: post #108
krunck wrote: post #46

The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.

I think this would be good for the people living in overcrowded, polluted and dirty cities in the world (let's be honest, they actually exist regardless of how that happened or why). Maybe they can unplug from 500+ AQI pollution and spend time with their loved ones and friends in a simulated clean world? Imagine working for 10-12 hours a day, and you come home to a pod (and a building could house thousands of pods, p…

Sounds similar to the black mirror ep https://en.wikipedia.org/wiki/Fifteen_Million_Merits

Re: Project Genie: Experimenting with infinite, interactive worlds

#175

Post by hmry »

seedie wrote: post #56
phailhaus wrote: post #27

I have no idea why Google is wasting their time with this. Trying to hallucinate an entire world is a dead-end. There will never be enough predictability in the output for it to be cohesive in any meaningful way, by design. Why are they not training models to help write games instead? You wouldn't have to worry about permanence and consistency at all, since they would be enforced by the code, like all games today. Lo…

Imo they explain pretty well what they are trying to achieve with SIMA and Genie in the Google Deepmind Podcast[1]. They see it as the way to get to AGI by letting AI agents learn for themselves in simulated worlds. Kind of like how they let AlphaGo train for Go in an enormous amount of simulated games. [1] https://youtu.be/n5x6yXDj0uo

"I need to go to the kitchen, but the door is closed. Easy. I'll turn around and wait for 60 seconds." -AI agent trained in this kind of world

Re: Project Genie: Experimenting with infinite, interactive worlds

#176

Post by phailhaus »

whalee wrote: post #159
phailhaus wrote: post #27

I have no idea why Google is wasting their time with this. Trying to hallucinate an entire world is a dead-end. There will never be enough predictability in the output for it to be cohesive in any meaningful way, by design. Why are they not training models to help write games instead? You wouldn't have to worry about permanence and consistency at all, since they would be enforced by the code, like all games today. Lo…

This was a common argument against LLMs, that the space of possible next tokens is so vast that eventually a long enough sequence will necessarily decay into nonsense, or at least that compounding error will have the same effect. Problem is, that's not what we've observed to happen as these models get better. In reality there is some metaphysical coarse-grained substrate of physics/semantics/whatever[1] which these m…

> Problem is, that's not what we've observed to happen as these models get better

Eh? Context rot is extremely well known. The longer you let the context grow, the worse LLMs perform. Many coding agents will pre-emptively compact the context or force you to start a new session altogether because of this. For Genie to create a consistent world, it needs to maintain context of everything, forever. No matter how good it gets, there will always be a limit. This is not a problem if you use a game engine and code it up instead.

Re: Project Genie: Experimenting with infinite, interactive worlds

#177

Post by MillionOClock »

elfly wrote: post #103
MillionOClock wrote: post #60

Earlier quoted context omitted.

An hybrid approach could maybe work, have a more or less standard game engine for coherence and use this kind of generative AI more or less as a short term rendering and physics sim engine.

I've thought about this same idea but it probably gets very complicated. Let's say, you simulate a long museum hallway with some vases in it. Who holds what? The basic game engine has the geometry, but once the player pushes it and moves it, it needs to inform the engine it did, and then to draw the next frame, read from the engine first, update the position in the video feed, then again feed it back to the engine. W…

Then maybe the engine should be less about really simulating the 3D world and just trying best to preserve consistency, more about providing memory and saving context for consistency than truly simulating a lot besides higher level concerns (at which point we might wonder if it couldn't be directly part of the model somehow), but writing those lines I realize there would probably still be many edge cases exactly like what you are describing...

Re: Project Genie: Experimenting with infinite, interactive worlds

#178

Post by boogrpants »

jplusequalt wrote: post #143
boogrpants wrote: post #138

Earlier quoted context omitted.

The American public is no different than an American corporation; trying to extract as much allegiance and loyalty as possible for as little compensation as possible. Your neighbors in the street protesting for comprehensive single payer healthcare? Yeah they're perfectly fine leaving your existence up to "market forces". Copy-paste office workers everywhere reciting memorized platitudes and compliance demands. You'r…

>I think we'll eventually get to the point where these are real time and have consistent representations You have a dangerously low opinion of your fellow man, and while I sympathize with your frustration, I would humbly suggest you direct that anger at owners of companies/politicians, rather than aim it at your everyday citizen.

[dead]

Re: Project Genie: Experimenting with infinite, interactive worlds

#180

Post by ceejayoz »

oceanplexian wrote: post #163
in-silico wrote: post #52

Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.

Yeah and the goal of Instagram was to share quirky pictures you took with your friends. Now it’s a platform for influencers and brainrot; arguably it has done more damage than drugs to younger generations. As soon as this thing is hooked up to VR and reaches a tipping point with the general public we all know exactly what is going to happen. The creation of the most profitable, addictive and ultimately dystopian tech…

The good news is we’ll finally have an answer for the Fermi Paradox.
Post reply on HN