Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Genie 3: A new frontier for world models

deepmind.google

Post reply on HN
Print view View original

441–450 of 512 posts

Re: Genie 3: A new frontier for world models

#441

Post by kiririn7 »

tim333 wrote: post #380
timeattack wrote: post #75

Advances in generative AI are making me progressively more and more depressive. Creativity is taken from us at exponential rate. And I don't buy argument from people who are saying they are excited to live in this age. I can get that if that technology stopped at current state and remained to be just tools for our creative endeavours, but it doesn't seem to be an endgame here. Instead it aims to be a complete replace…

>So what is final state here for us? The merge. ( https://blog.samaltman.com/the-merge ) I'm quite enthusiastic. I've always thought mortality sucks.

agree about mortality but im afraid the idea of merging may be naive

Re: Genie 3: A new frontier for world models

#442

Post by wolttam »

abrookewood wrote: post #432
wolttam wrote: post #388

Earlier quoted context omitted.

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

Can you explain why this is the case? I don't understand why.

I'll try! Let's consider a tree blowing in the wind:

To classically render this in any realistic fashion, it quickly gets complex. Between the physics simulation (rather involved) and the number of triangles (trees have many branches and leaves), you're going to be doing a lot of math.

I'll emphasize "realistic" - sure, we can real-time render trees in 2025 that look.. ok. However, take more than a second to glance at it and you will quickly start to see where we have made compromises to the tree's fidelity to ensure it renders at an adequate speed on contemporary hardware.

Now consider a world model trained on enough tree footage that it has gained an "intuition" about how trees look and behave. This world model doesn't need to actually simulate the entire tree to get it to look decent.. it can instead directly output the pixels that "make sense". Much like a human brain can "simulate" the movement of an object through space without expending much energy - we do it via prediction based on a lot of training data, not by accurately crunching a bunch of numbers.

That's just one tree, though - the real world has a lot of fidelity to it. Fidelity that would be extremely expensive to simulate to get a properly realistic output on the other side.

Instead we can use these models which have an intuition for how things aught to look. They can skip the simulation and just give you the end result that looks passable because it's based on predictions informed by real-world data.

Re: Genie 3: A new frontier for world models

#444

Post by fc417fc802 »

casenmgreen wrote: post #162
diwank wrote: post #70

Earlier quoted context omitted.

> Future robots may learn in their dreams... So prescient. I definitely think this will be a thing in the near future ~12-18 months time horizon

I may be wrong, but this seems to make no sense. A neural net can produce information outside of its original data set, but it is all and directly derived from that initial set. There are fundamental information constraints here. You cannot use a neural net to itself generate from its existing data set wholly new and original full quality training data for itself. You can use a neural net to generate data, and you ca…

Approximately speaking, you have a world model and an agent model. You continue to train the world model using data collected by the robot day-to-day. The robot "dreams" by running the agent model against the world model instead of moving around in the real world. Dreaming for thousands of (simulated) hours is much more efficient than actually running the physical hardware for thousands of wall clock hours.

Re: Genie 3: A new frontier for world models

#445

Post by frabcus »

svdr wrote: post #368
rotexo wrote: post #172

Earlier quoted context omitted.

You know that thing in anxiety dreams where you feel very uncoordinated and your attempts to manipulate your surroundings result in unpredictable consequences? Like you try to slam on the brake pedal but your car doesn’t slow down, or you’re trying to get a leash on your dog to lead it out of a dangerous situation and you keep failing to hook it on the collar? Maybe that’s extra latency because your brain is trying t…

Your brain does not need to render any environments, just the experience of being in them.

Firstly it can render environments in detail. I'm (mostly) aphantasic even in dreams, so this wasn't obvious to me. But most people literally get visual renderings in their mind.

Secondly, it's fairly clear now that our sensory inputs are not being experienced as sensory inputs. We experience a reconstruction. Obvious basic sign of this is that we fill in the gap in vision where the optic nerve is. But generally, we're making an integrated world model all the time out of the senses, and are conscious of that world model.

You're right though, both the above are rendering the experience and can take shortcuts for that. It's sufficiently detailed in each case though that it kinda is rendering the world too, in some sense.

Re: Genie 3: A new frontier for world models

#447

Post by fc417fc802 »

CharlieDigital wrote: post #320
pantalaimon wrote: post #87

Earlier quoted context omitted.

Joscha Bach postulates that what we call consciousness must be something rather simple, an emergent property present in all sufficiently complex biological organisms. We don't inherit any software, so cognitive function must bootstrap itself from it's underlying structure alone. https://media.ccc.de/v/38c3-self-models-of-loving-grace

> We don't inherit any software I wonder, though. Many animal species just "know" how to perform certain complex actions without being taught the way humans have to be taught. Building a nest, for example. If you say that this is emergent from the "underlying structure alone", doesn't this mean that it would still be "inherited" software (though in this case, maybe we think of it like punch cards).

A similar phenomenon was demonstrated with deep neural networks nearly a decade ago. You optimize the architecture using randomized weights instead of optimizing the weights. You can still optimize the weights in a separate additional step to improve performance.

Re: Genie 3: A new frontier for world models

#448

Post by energy123 »

lossolo wrote: post #402
wolttam wrote: post #388

Earlier quoted context omitted.

At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges

> At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, This is already happening to some extent, some games struggle to reach 60 FPS at 4K resolution with maximum graphics settings using traditional rasterization alone, so technologies like DLSS 3 frame generation are used to improve performance.

Instead of the binary of traditional games vs AI, it's worth thinking more about hybrids.

You could have a stripped down traditional game engine, but without any rendering, that gives a richer set of actions to the neural net. Along with some asset hints, story, a database (player/environment state) the AI can interact with, etc. The engine also provides bounds and constraints.

Basically, we need to work out the new boundary between engine and AI. Right now it's "upsample and interpolate frames", but as AI gets better, what does that boundary become?

Re: Genie 3: A new frontier for world models

#449

Post by scoot »

qingcharles wrote: post #35
NoScopeNinja wrote: post #11

It sounds cool that Genie 3 can make whole worlds you can explore, but I wonder how soon regular people will actually get to try it out?

These guys are working on the same thing and have a real demo you can play: https://odyssey.world/introducing-interactive-video

I'm pretty sure that got debunked as a handful of overfitted examples that fall apart as soon as you leave the area that was 3D recorded. Google's new tech is on another level entirely.

Re: Genie 3: A new frontier for world models

#450

Post by tripzilch »

jonas21 wrote: post #391
glenstein wrote: post #126

Earlier quoted context omitted.

>It's basically what every major AI lab head is saying from the start. I suppose it depends what you count as "the start". The idea of AI as a real research project has been around since at least the 1950s. And I'm not a programmer or computer scientist, but I'm a philosophy nerd and I know debates about what computers can or can't do started around then. One side of the debate was that it awaited new conceptual and…

When they say "the start", I think they mean the start of the current LLM era (circa 2017). The main story of this time has been a rejection of the idea that major conceptual breakthroughs and complex architectures are needed to achieve intelligence. Instead, it's better to focus on simple, general-purpose methods that can scale to massive amounts of data and compute (i.e. the Bitter Lesson [1]). [1] http://www.incom…

Oof ... to call other people's decades of research into directed machine learning "a colossal waste of researcher's time" is indeed a rather toxic point of view unsurprisingly causing a bitter reaction in scientists/researchers.

Even if his broader point might be valid (about the most fruitful directions in ML), calling something a "bitter lesson" while insulting a whole field of science is ... something.

Also as someone involved in early RL, he should know better.

Post reply on HN