Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Genie 3: A new frontier for world models

deepmind.google

Post reply on HN
Print view View original

401–410 of 512 posts

Re: Genie 3: A new frontier for world models

#401

Post by blibble »

ollin wrote: post #131
ollin wrote: post #38

This is very encouraging progress, and probably what Demis was teasing [1] last month. A few speculations on technical details based on staring at the released clips: 1. You can see fine textures "jump" every 4 frames - which means they're most likely using a 4x-temporal-downscaling VAE with at least 4-frame interaction latency (unless the VAE is also control-conditional). Unfortunately I didn't see any real-time foo…

Regarding latency, I found a live video of gameplay here [1] and it looks like closer to 1.1s keypress-to-photon latency (33 frames @ 30fps) based on when the onscreen keys start lighting up vs when the camera starts moving. This writeup [2] from someone who tried the Genie 3 research preview mentions that "while there is some control lag, I was told that this is due to the infrastructure used to serve the model rath…

> I found a live video of gameplay here [1] and it looks like closer to 1.1s keypress-to-photon latency (33 frames @ 30fps) based on when the onscreen keys start lighting up vs when the camera starts moving.

so better than Stadia?

Re: Genie 3: A new frontier for world models

#402

Post by lossolo »

wolttam wrote: post #388
maerF0x0 wrote: post #376

I'm still struggling to imagine a world where predicting the next pixel wins over over building a deterministic thing that is then ran. Eg: Using AI to generate textures, wire models, motion sequences which themselves sum up to something that local graphics card can then render into a scene. I'm very much not an expert in this space, but to me it seems if you do that, then you can tweak the wire model, the texture, m…

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

> At some point it will be computationally cheaper to predict the next pixel than to classically render the scene,

This is already happening to some extent, some games struggle to reach 60 FPS at 4K resolution with maximum graphics settings using traditional rasterization alone, so technologies like DLSS 3 frame generation are used to improve performance.

Re: Genie 3: A new frontier for world models

#403

Post by roywiggins »

HeatrayEnjoyer wrote: post #359
roywiggins wrote: post #270

Earlier quoted context omitted.

My prediction is that personal generation is going to be niche forever, for purely social reasons. The demand for fandoms and fan communities seems to be essentially unlimited. Big artists have big fandoms, tiny ones have tiny fandoms, but none of that works with personalized generations.

Communities around fictional universes are already fractured and shrinking in member size because of the sheer number of algorithmically targeted universes available. Water cooler talk about what happened this week in M.A.S.H. or Friends is extinct. Worse, in the long run even community may be synthesized. If a friend is meat or if they're silicon (or even carbon fiber!), does it matter if you can't tell the differen…

I think things will look a lot more like Vinge's Rainbows End than everyone burrowing into their own personal algoentertainment. I can't speak for GenZ but when D&D can sell out Madison Square Garden, there doesn't seem to be any softening in people's interest in fandom.

Virtual influencers might be a big thing, Hatsune Miku has lots of fans. But it's still a shared fandom.

Re: Genie 3: A new frontier for world models

#404

Post by [deleted] »

brap wrote: post #393
wolttam wrote: post #388

Earlier quoted context omitted.

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

I imagine a future where the “high level” stuff in the environment is pre defined by a human (with or without assistance from AI), and then AI sort of fills in the blanks on the fly. So for example, a game designer might tell the AI the floor is made of mud, but won’t tell the AI what it looks like if the player decides to dig a 10 ft hole in the mud, or how difficult it is to dig, or what the mud sounds like when th…

[deleted]

Re: Genie 3: A new frontier for world models

#405

Post by corimaith »

kkukshtel wrote: post #221
modeless wrote: post #13

Consistency over multiple minutes and it runs in real time at 720p? I did not expect world models to be this good yet. > Genie 3’s consistency is an emergent capability So this just happened from scaling the model, rather than being a consequence of deliberate architecture changes? Edit: here is some commentary on limitations from someone who tried it: https://x.com/tejasdkulkarni/status/1952737669894574264 > - Physi…

I similarly am surprised at how fast they are progressing. I wrote this piece a few months ago about how I think steering world model output is the next realm of AAA gaming: https://kylekukshtel.com/diffusion-aaa-gamedev-doom-minecraf... But even when I wrote that I thought things were still a few years out. I facetiously said that Rockstar would be nerd-sniped on GTA6 by a world model, which sounded crazy a few mont…

The future of games was MMORPGs and RPG-ization in general as other genres adopted progression systems. But the former two are simply too expensive and risky even today for AAA to develop. Which brings us to another point, the problem with Western AAA is more about high levels of risk aversion, which is what's really feeding the lack of imaginative. And that's more to do with the economics of opportunity cost to the S&P 500.

Anyways, crafting pretty looking worlds is one thing, but you still need to fill them in with something worth doing, and that's something we haven't really figured out. That's one of the reasons why the sandbox MMORPG was developed as opposed to "themeparks". The underlying systems, the backend is the real meat here. At most with the world models right now is that you're replacing 3d artists and animators, but I would not say that is a real bottleneck in relation to one's own limitations.

Re: Genie 3: A new frontier for world models

#406

Post by tomrod »

HeatrayEnjoyer wrote: post #362
tomrod wrote: post #309

Earlier quoted context omitted.

Branding and differentiation. People still value Amish furniture or woodworking despite Ikea existing. I love that if I want a cheap chair made of cardboard and glue that I can find something to satisfy that need; but I still buy nice furniture when I can. AI creations are analogous. I've seen some cool AI stuff, but it definitely doesn't replace the real "organic" art one finds.

What if it's not cardboard and glue but woodworking of ultra-master quality? These fears aren't realized if AI never achieves superhuman performance, but what if they do?

(1) It's not, currently, and isn't on the horizon despite current polish with LLM frameworks.

(2) AI has already achieved superhuman performance in breadth and, with tuning, depth.

Re: Genie 3: A new frontier for world models

#407

Post by ProofHouse »

Can anyone specifically working or with expertise in this field, give even a best guest breakdown (or better) of the technology and architecture, system design or possibly even the compute requirement's of how they think this was implemented? Very curious as to how thing works and methods employed, as they are atm tight lipped generally. So kind of curious for those who are specialists in this space what they could surmise or speculate on the implementation of Genie 3

Re: Genie 3: A new frontier for world models

#408

Post by corimaith »

lossolo wrote: post #402
wolttam wrote: post #388

Earlier quoted context omitted.

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

> At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, This is already happening to some extent, some games struggle to reach 60 FPS at 4K resolution with maximum graphics settings using traditional rasterization alone, so technologies like DLSS 3 frame generation are used to improve performance.

I think that's more to do with poor optimization that the actual level of graphical fidelity that requires it.

Re: Genie 3: A new frontier for world models

#409

Post by ProofHouse »

modeless wrote: post #13

Consistency over multiple minutes and it runs in real time at 720p? I did not expect world models to be this good yet. > Genie 3’s consistency is an emergent capability So this just happened from scaling the model, rather than being a consequence of deliberate architecture changes? Edit: here is some commentary on limitations from someone who tried it: https://x.com/tejasdkulkarni/status/1952737669894574264 > - Physi…

Curious how multiplayer would possibly work not only logistically, but technically and from a game play POV

Re: Genie 3: A new frontier for world models

#410

Post by energy123 »

maerF0x0 wrote: post #376

I'm still struggling to imagine a world where predicting the next pixel wins over over building a deterministic thing that is then ran. Eg: Using AI to generate textures, wire models, motion sequences which themselves sum up to something that local graphics card can then render into a scene. I'm very much not an expert in this space, but to me it seems if you do that, then you can tweak the wire model, the texture, m…

"Wins" in the sense of being useful, or being on the critical path to AGI?
Post reply on HN