Live data from Hacker News

Video generation models as world simulators

openai.com

111–120 of 171 posts

Re: Video generation models as world simulators

#111

Earlier quoted context omitted.

Sounds like simulation theory is closer and closer to being proven.

Except there is always an original at the root. There’s no way to prove that’s not us.

Something something linked list with loop.

Re: Video generation models as world simulators

#112
post #15
post #2

I find it wild that this model does not have explicit 3D prior, yet learns to generate videos with such 3D consistency, you can directly train a 3D representation (NeRF-like) from those videos: https://twitter.com/BenMildenhall/status/1758224827788468722

I wonder how much it'd improve if trained on stereo image data.

Moving camera is just stereo.

Re: Video generation models as world simulators

#113

I like that this one shows some "fails", and not just the top of the top results: For example, the surfer is surfing in the air at the end: https://cdn.openai.com/tmp/s/prompting_7.mp4 Or this "breaking" glass that does not break, but spills liquid in some weird way: https://cdn.openai.com/tmp/s/discussion_0.mp4 Or the way this person walks: https://cdn.openai.com/tmp/s/a-woman-wearing-a-green-dress-a... Or wherever…

Where do you find the last two?

Part of this website changes after the video finished and switches to the next video. There is no way to control it. These are both "X wearing Y taking a pleasant stroll in Z during W"

Re: Video generation models as world simulators

#114

Earlier quoted context omitted.

But it starts to make sense, when you think about the fact that Minecraft is owned by Microsoft. example video links from TFA: https://cdn.openai.com/tmp/s/simulation_6.mp4 https://cdn.openai.com/tmp/s/simulation_7.mp4

I don't think that has anything to do with it. There's just millions of hours of minecraft video on youtube and twitch.

But If they wouldn't have the ok from Microsoft there would be a chance that they start talking about copyright infringement.

Re: Video generation models as world simulators

#115

Earlier quoted context omitted.

V-Jepa is actually super impressive. I have nothing but respect for Yann LeCun & his team, they really have been on a rampage lately.

One more French brain we didn't manage to keep. The drain is just crazy at this point.

What would fix the issue?

Re: Video generation models as world simulators

#117

Earlier quoted context omitted.

Except there is always an original at the root. There’s no way to prove that’s not us.

The root world can spawn many simulations and simulations can be spawned within simulations. It becomes far more likely that we exist in a simulated world than in the root world.

This argument always bothers me.

The probability of being in any simulation is conditional on the one above, which necessarily decreases exponentially. Any simulation running an equivalent simulation will do so much slower, so you get a geometric series of degrading probabilities.

The rate of decay will be massive, imagine how long and how much resources it would take us to simulate our universe, even in a hand waved AI+lazy compute way that also spawns subconsciousnesses. The inverse of that is the sequences ratio.

So even in theory, the probability of you being in any of the simulated universes is P(we can simulate a universe) / (1-1/time to simulate) - prob we’re in the top universe.

Thinking this probability is overwhelming because of the nesting effect is false.

Re: Video generation models as world simulators

#118

I think people might be missing what this enables. It can make plausible continuations of video, with realistic physics. What happens if this gets fast enough to work _in real time_. Connect this to a robot that has a real time camera feed. Have it constantly generate potential future continuations of the feed that it's getting -- maybe more than one. You have an autonomous robot building a real time model of the wor…

A 3d model with object permanency is definitely a step in the right direction of something or other but for clarity let us dial back down the level of graphical detail. A Pacman bot is not AGI. Might get it to eat all the dots correctly where as before if something scrolled off the screen it'd forget about it and glitch out - but you didn't fan any flames of consciousness into existence as of yet.

Is a human that manages to eat all the skittles and walk without falling into deadly holes AGI? Why?

Re: Video generation models as world simulators

#119

I think people might be missing what this enables. It can make plausible continuations of video, with realistic physics. What happens if this gets fast enough to work _in real time_. Connect this to a robot that has a real time camera feed. Have it constantly generate potential future continuations of the feed that it's getting -- maybe more than one. You have an autonomous robot building a real time model of the wor…

So basically a brain in a vat, reality as we experience it, our thoughts as prompts.

Re: Video generation models as world simulators

#120

Earlier quoted context omitted.

That is creepy...

Take a look at these adorable kangaroos to relax: https://cdn.openai.com/tmp/s/an-adorable-kangaroo-wearing-bl... https://cdn.openai.com/tmp/s/an-adorable-kangaroo-wearing-a-...

Is the guy with the arm sticking out of his shoulder the relaxing part? :-)
Post reply on HN