Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Genie: Experimenting with infinite, interactive worlds

blog.google

Post reply on HN
Print view View original

31–40 of 347 posts

Re: Project Genie: Experimenting with infinite, interactive worlds

#31

Post by api »

ofrzeta wrote: post #22

I don't know ... it's impressive and all but the result always looks kind of dead.

It's super cool but I see it as a much more flexible open ended take on the idea of procedurally generated worlds where hard-coded deterministic math and rendering parameters are replaced by prompt-able models.

The deadness you're talking about is there in procedural worlds too, and it stems from the fact that there's not actually much "there." Think of it as a kind of illusion or a magic trick with math. It replicates some of the macro structure of the world but the true information content is low.

Search YouTube for procedural landscape examples. Some of them are actually a lot more visually impressive than this, but without the interactivity. It's a popular topic in the demo scene too where people have made tiny demos (e.g. under 1k in size) that generate impressive scenes.

I expect to see generative AI techniques like this show up in games, though it might take a bit due to their high computational cost compared to traditional procedural generation.

Re: Project Genie: Experimenting with infinite, interactive worlds

#32

Post by observationist »

halfmatthalfcat wrote: post #30
observationist wrote: post #26

Earlier quoted context omitted.

LeCun wasn't producing results. He was obstinate and insistent on his own theories and ideas which weren't, and possibly aren't, going anywhere. He refused to engage with LLMs and compete in the market that exists, and spent all his effort and energy on unproven ideas and research, which split the company's mission and competitiveness. They lost their place as one of the top 4 AI companies, and are now a full generat…

Were you there or just an attentive outsider?

Attentive outsider and acquaintance of a couple people who are or were employed there. Nothing I'm saying is particularly inside baseball, though, it's pretty well covered by all the blogs and podcasts.

Re: Project Genie: Experimenting with infinite, interactive worlds

#33

Post by WarmWash »

The actual breakthrough with Genie is being able to turn around and look back, and seeing the same scene that was there before. A few other labs have similar world simulators, but they all struggle badly with keeping coherence of things not in view. Hence why they always walk forwards and never look around.

Re: Project Genie: Experimenting with infinite, interactive worlds

#34

Post by qwertox »

sy26 wrote: post #10

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

Isn't it more like this: JEPA looks at the video, "a dog walks out of the door, the mailman comes, dog is happy" and the next frame would need to look like "mailman must move to mailbox, dog will run happily towards him", which then an image/video generator would need to render.

Genie looks at the video, "when this group of pixels looks like this and the user presses 'jump', I will render the group different in this way in the next frame."

Genie is an artist drawing a flipbook. To tell you what happens next, it must draw the page. If it doesn't draw it, the story doesn't exist.

JEPA is a novelist writing a summary. To tell you what happens next, it just writes "The car crashes." It doesn't need to describe what the twisted metal looks like to know the crash happened.

Re: Project Genie: Experimenting with infinite, interactive worlds

#35

Post by saberience »

ofrzeta wrote: post #22

I don't know ... it's impressive and all but the result always looks kind of dead.

This reminds me of the comments by programmers roughly two years ago:

"Sure it can write a single function but the code is terrible when it tries to write a whole class..."

Re: Project Genie: Experimenting with infinite, interactive worlds

#36

Post by WarmWash »

bdbdbdb wrote: post #21
0xcb0 wrote: post #7

I keep on repeating myself, but it feels like I'm living in the future. Can't wait to hook this up to my old Oculus glasses and let Genie create a fully realistic sailing simulator for me, where I can train sailing with realistic conditions. On boats I'd love to sail. If making games out of these simulations work, it't be the end for a lot of big studios, and might be the renaissance for small to one person game stud…

It's very impressive tech but subject to the same limitations as other generative AI: Inconsistency, inaccurate physics, limited time, lag, massively expensive computation. You COULD create a sailing sim but after ten minutes you might be walking on water, or in the bath, and it would use more power than a small ferry. There's no way this tech can run on a PS5 or anything close to it.

Five years is nothing to wait for tech like this. I'm sure we will see the first crop of, however small, "terminally plugged in" humans on the back of this in the relatively near future.

Re: Project Genie: Experimenting with infinite, interactive worlds

#37

Post by saberience »

hiccuphippo wrote: post #18
srameshc wrote: post #9

What’s the endgame here? For a small gaming studio, what are the actual implications?

It seems to be generating images in real time, not 3d scenes. It might still be useful for prototyping.

There are collisions though and physics seemingly, so it doesn't seen to be a huge stretch that this could be used for games.

Re: Project Genie: Experimenting with infinite, interactive worlds

#38

Post by Avicebron »

0xcb0 wrote: post #7

I keep on repeating myself, but it feels like I'm living in the future. Can't wait to hook this up to my old Oculus glasses and let Genie create a fully realistic sailing simulator for me, where I can train sailing with realistic conditions. On boats I'd love to sail. If making games out of these simulations work, it't be the end for a lot of big studios, and might be the renaissance for small to one person game stud…

Honestly getting a Sunfish is probably cheaper than the a VR headset if you want to "train sailing"

Re: Project Genie: Experimenting with infinite, interactive worlds

#39

Post by [deleted] »

qwertox wrote: post #34
sy26 wrote: post #10

I have been confused for a long time why FB is not motivated enough to invest in world models, it IS the key to unblock their "metaverse" vision. And instead they let go Yann LeCun.

Isn't it more like this: JEPA looks at the video, "a dog walks out of the door, the mailman comes, dog is happy" and the next frame would need to look like "mailman must move to mailbox, dog will run happily towards him", which then an image/video generator would need to render. Genie looks at the video, "when this group of pixels looks like this and the user presses 'jump', I will render the group different in this…

[deleted]

Re: Project Genie: Experimenting with infinite, interactive worlds

#40

Post by mediaman »

xyzsparetimexyz wrote: post #19
srameshc wrote: post #9

What’s the endgame here? For a small gaming studio, what are the actual implications?

It means you should go the other way. Open world winning against smaller, handcrafted environments and stories was generally a mistake, and so is this.

What does it mean, that open world winning was a mistake? That the market is wrong, and peoples' preferences were incorrect, and they should prefer small handcrafted environments instead of what they seem to actually buy?
Post reply on HN