Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

331–340 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#331
post #167

Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…

yes! obligatory Karl Friston reference... https://www.youtube.com/watch?v=mU4qdFgPT0A

Do jump forward to the contents' discussion marker unless you enjoy British professor banter.

Re: Project Genie: Experimenting with infinite, interactive worlds

#332
post #291
post #288

Earlier quoted context omitted.

“ This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter.” This is an orthodox position in modern philosophy, dating back to…

It’s the Allegory of the Cave, isn’t it?

In my amateur opinion, it's almost the opposite. For Plato, the material world, while "real" enough, is less important and in some sense less True than the higher immaterial world of Forms or Ideas. The highest, truest, realest world is "above" this one, related to cognition, and (more or less) accessible by reason. We may be in a cave, but all we have to do is walk up into the sunlight — which, by the way, is nothing but a higher and truer form of light than our current firelight. (This idea that material objects partake of their corresponding higher-level Ideas leads to the Third Man paradox: if it is the Form of Man that compasses similar material instances such as Socrates and Achilles, is there then a third... thing... that compasses Socrates, Achilles, and Man itself?)

For Kant, and therefore for Schopenhauer, the visible world is composed merely of objects, which are by definition only mental representations: a world of objects "exists" only in the mind of a subject. If there is a Thing-in-Itself (which even Kant does not doubt, if I recall correctly), it certainly cannot be a mental representation: the nature of the Thing-in-Itself is unknowable (says Kant) but certainly in no way at all like the mere object that appears to our mental processes. (Schopenhauer says the Thing-in-Itself is composed of pure Will, whatever that means.) The realest world is "behind" or "below" the visible one, completely divorced from human reason, and by definition completely inaccessible to any form of cognition (which includes the sensory perception we share with the animals, as well as the reason that belongs to humans alone). The Third Man paradox makes no sense at all for Kant, first because whatever the ineffable Thing-in-Itself is, it certainly won't literally "partake" of any mental concept we might come up with, and secondly because it would be a category error to suppose that any property could be true of both a mental object and a thing-in-itself, which are nothing alike. (The Thing-in-Itself doesn't even exist in time or space, nor does it have a cause. Time, space, and causality are all purely human frameworks imposed by our cognitive processes: to suppose that space has any real existence simply because you perceive it is, again, a category error, akin to supposing that the world is really yellow-tinged just because you happen to be wearing yellow goggles.)

Re: Project Genie: Experimenting with infinite, interactive worlds

#333

Earlier quoted context omitted.

But how does it interpolate?

Pixel by pixel, time-slice by time-slice, in a 2D+T convolution. You provide enough examples of videos of changing point-of-view, and the model reproduces what it is given.

Yes, it reproduces what it is given by modelling the rules of physics, geometry, etc.

For example, image generators like stable diffusion carry strong representations of depth and geometry, such that performant depth estimation models can be built out of them with minimal retraining. This continues to be true for video generation models.

Early work on the subject: https://arxiv.org/pdf/2409.09144

Re: Project Genie: Experimenting with infinite, interactive worlds

#334
post #288

Earlier quoted context omitted.

This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter. This is one of the core tenets of Buddhism, and it's also expounded o…

“ This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter.” This is an orthodox position in modern philosophy, dating back to…

Indeed, it's been great learning about the various interpretations as this idea took hold over the years!

What I wasn't able to properly highlight is how this belief has become a fundamental part of my day to day, moment to moment experience. I enjoy the constant and absolute knowledge that everything that's happening is my interpretation. And it gives me a superpower -- because for most of my life the world felt unforgiving and unpredictable. But it's actually the complete opposite, since whatever we interpret is actually in our control.

I also credit my understanding of this as a reality vs an intellectual concept to Siddhartha Gautam and his presentation of "samsara". But wherever it comes from, it is an inescapable idea and I encourage all HNers to dive deeper.

Re: Project Genie: Experimenting with infinite, interactive worlds

#335

Earlier quoted context omitted.

This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter. This is one of the core tenets of Buddhism, and it's also expounded o…

Thank you for linking this! I'm a big fan of Egan but had never read this particular short story. I feel like Egan is perhaps the only contemporary author who actually _gets_ consciousness.

He's the best hard sci fi author by far. Permutation City is also a must read especially because of its thought experiments around consciousness and computation.

Re: Project Genie: Experimenting with infinite, interactive worlds

#336

Earlier quoted context omitted.

This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter. This is one of the core tenets of Buddhism, and it's also expounded o…

I'm not sure this is unique to consciousness (whatever that is). What would it even mean to directly interact with the physical world? Even the most precise scientific experiments are a series of indirect measurements of something that perhaps in some sense is fundamentally unknowable.

That's a fair point. I envision this as the difference between an electron interacting with the proton or an acid reacting with the base vs me touching my keyboard. Do you feel there's a difference there?

Re: Project Genie: Experimenting with infinite, interactive worlds

#337

Earlier quoted context omitted.

This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter. This is one of the core tenets of Buddhism, and it's also expounded o…

> We are never able to interact with the physical world directly What would count as anything or anyone interacting with the physical world directly?

My thoughts on that https://news.ycombinator.com/item?id=46832714

Re: Project Genie: Experimenting with infinite, interactive worlds

#338

Earlier quoted context omitted.

I'm not sure this is unique to consciousness (whatever that is). What would it even mean to directly interact with the physical world? Even the most precise scientific experiments are a series of indirect measurements of something that perhaps in some sense is fundamentally unknowable.

That's a fair point. I envision this as the difference between an electron interacting with the proton or an acid reacting with the base vs me touching my keyboard. Do you feel there's a difference there?

I don't know what consciousness is or how to talk about it. But at a basic level I don't believe there is a category of phenomena separate from the physical world or inaccessible to the scientific method; and if there were I don't know how you would say anything meaningful about them. I am however willing to believe that rocks are not conscious.

Re: Project Genie: Experimenting with infinite, interactive worlds

#339

SPIN THE CAMERA 1080 DEGREES YOU COWARDS The only test I ever want to see with these frame-gen models is a full 1080 degree camera spin. Miss me with that 30 degree back and forth crap. I want multiple full turns. Some jitter and a little back-and-forth wobble is fine. But I want multiple full spins. I’m pretty sure I know why they don’t do this :)

> SPIN THE CAMERA 1080 DEGREES YOU COWARDS > a full 1080 degree camera spin Do you mean 3 full turns, or do you mean 180 (one half-turn)?

I mean three full turns. I promise it will fail catastrophically.
Post reply on HN