Live data from Hacker News

Genie 3: A new frontier for world models

deepmind.google

381–390 of 512 posts

Re: Genie 3: A new frontier for world models

#381
I wish they would share more about how it works. Maybe a reseach paper for once? we didn't even get a technical report.

From my best guess: it's a video generation model like the ones we already head. But they condition inputs (movement direction, viewangle). Perhaps they aren't relative inputs but absolute and there is a bit of state simulation going on? [although some demo videos show physics interactions like bumping against objects - so that might be unlikely, or maybe it's 2D and the up axis is generated??].

It's clearly trained on a game engine as I can see screenspace reflection artefacts being learned. They also train on photoscans/splats... some non realistic elements look significantly lower fidelity too..

some inconsistencies I have noticed in the demo videos:

- wingsuit discollcusions are lower fidelity (maybe initialized by high resolution image?)

- garden demo has different "geometry" for each variation, look at the 2nd hose only existing in one version (new "geometry" is made up when first looked at, not beforehand).

- school demo has half a caroutside the window? and a suspiciously repeating pattern (infinite loop patterns are common in transformer models that lack parameters, so they can scale this even more! also might be greedy sampling for stability)

- museum scene has odd reflection in the amethyst box, like the rear mammoth doesn't have reflections on the right most side of the box before it's shown through the box. The tusk reflection just pops in. This isn't fresnel effect.

Re: Genie 3: A new frontier for world models

#383

Earlier quoted context omitted.

I see two ways this is going to go: 1. Universal Basic Income as we're on the way to a post-scarcity society. Unlikely to actually happen due to greed. 2. We take inspiration from the french revolution and then return to a simpler time.

> Unlikely to actually happen due to greed. Greed makes no sense in a truly post scarcity society. There is no scarcity from which to take in a zero sum way from another. Status is the real issue. Humans use status to select sexually, and the display is both competitive and comparative. It doesnt matter absolutely how many pants you have, only that you have more and better than your competition. I actually think this…

I think UBI is likely to happen because of greed - people like free stuff and will vote for it is it's real. The trouble with the pitch:

>Vote for me and we'll hand free money to everyone and the robots will do the work

at the moment is the robots doing the work don't exist. Things will change when they do.

Re: Genie 3: A new frontier for world models

#384

Earlier quoted context omitted.

And made hands 10x worse. Now hands are good, text is good, image is good, so we’ll have to play where’s Waldo all over again trying to find the flaw. It’s going to eventually get to a point where it’s one of those infinite zoom videos where the AI watermark is the size of 1/3rd of a pixel. What I’d really love to see more of is augmented video. Like, the stormtrooper vlogs. Runway has some good stuff but man is it a…

someone mentioned physics. Which might be an interesting conundrum because an important characteristic of games is that some part of them is both novel and unrealistic. (They're less fun if they're too real)

It depends on the genre. Simulation “games” tend to love realism of simulation while providing an accelerated time. Others like you said, are more fun with plausible physics or physics that bend the rules a little bit. Sometimes, a game just about funky physics becomes a hit - Goat Simulator.

Walking/Running/Steps have already been solved pretty well with NN’s, but simulation of vehicle engines and vehicle physics have not. Not to my knowledge. I suspect iRacing would be extremely interested in such a model.

edit

I take it back, PINN’s are a thing and now I have a new rabbit hole…

Re: Genie 3: A new frontier for world models

#385
We were working towards this years ago with Doarama/Ayvri, and I remember fondly in 2018 an investor literally yelling at me that I didn't know what I was talking about and AI would never be able to do this. Less than a decade later, here we are.

Our product was a virtual 3d world made up of satellite data. Think of a very quick, higher-res version of google earth, but the most important bit was that you uploaded a GPS track and it re-created the world around that space. The camera was always focused on the target, so it wasn't a first person point of view, which, for the most part, our brains aren't very good at understanding over an extended period of time.

For those curious about the use case, our product was used by every paraglider in the world, commercial drone operations, transportation infrastructure sales/planning, out-door events promotions (specifically bike and ultramarathon races).

Though I suspect we will see a new form of media come from this. I don't pretend to suggest exactly what this media will be, but mixing this with your photos we can see the potential for an infinitely re-framable and zoomable type of photo media.

Creating any "watchable" content will be challenging if the camera is not target focused, and it makes it difficult to create a storyline if you can't dictate where the viewer is pointed.

Re: Genie 3: A new frontier for world models

#386

We were working towards this years ago with Doarama/Ayvri, and I remember fondly in 2018 an investor literally yelling at me that I didn't know what I was talking about and AI would never be able to do this. Less than a decade later, here we are. Our product was a virtual 3d world made up of satellite data. Think of a very quick, higher-res version of google earth, but the most important bit was that you uploaded a G…

To be fair, I'm seeing the demo video, and I still don't believe it's possible. This is sci-fi tech.

Re: Genie 3: A new frontier for world models

#387

The Simulation Theory presents the following trilemma, one of which must be true: 1. Almost all human-level civilizations go extinct before reaching a technologically mature “posthuman” stage capable of running high-fidelity ancestor simulations. 2. Almost no posthuman civilizations are interested in running simulations of their evolutionary history or beings like their ancestors. 3. We are almost certainly living in…

But there are lots of critiques of that supposed trilemma.

Why would beings in simulations be conscious?

Or maybe running simulations is really expensive and so it's done sometimes (more than "almost none") but only sometimes (nowhere near "we are almost certainly").

Or simulations are common but limited? You don't need to simulate a universe if all you want to do is simulate a city.

The "trilemma" is an extreme example of black-and-white thinking. In the real world, things cost resources and so there are tradeoffs -- so middle grounds are the rule, not extremes.

Re: Genie 3: A new frontier for world models

#388

I'm still struggling to imagine a world where predicting the next pixel wins over over building a deterministic thing that is then ran. Eg: Using AI to generate textures, wire models, motion sequences which themselves sum up to something that local graphics card can then render into a scene. I'm very much not an expert in this space, but to me it seems if you do that, then you can tweak the wire model, the texture, m…

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity.

The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

Re: Genie 3: A new frontier for world models

#389

Consistency over multiple minutes and it runs in real time at 720p? I did not expect world models to be this good yet. > Genie 3’s consistency is an emergent capability So this just happened from scaling the model, rather than being a consequence of deliberate architecture changes? Edit: here is some commentary on limitations from someone who tried it: https://x.com/tejasdkulkarni/status/1952737669894574264 > - Physi…

consider the hardware DOOM runs on. 720p would only be a true test of capability if every bit of possible detail was used.

Re: Genie 3: A new frontier for world models

#390

Advances in generative AI are making me progressively more and more depressive. Creativity is taken from us at exponential rate. And I don't buy argument from people who are saying they are excited to live in this age. I can get that if that technology stopped at current state and remained to be just tools for our creative endeavours, but it doesn't seem to be an endgame here. Instead it aims to be a complete replace…

"Creativity is taken from us at exponential rate"

Nothing is being taken away.

Post reply on HN