Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real-time, in-camera background compositing in The Mandalorian

ascmag.com

Post reply on HN
Print view View original

11–20 of 126 posts

Re: Real-time, in-camera background compositing in The Mandalorian

#11

Post by nocut12 »

ragebol wrote: post #4
cbhl wrote: post #2

"The solution was ... a dynamic, real-time, photo-real background played back on a massive LED video wall and ceiling ... rendered with correct camera positional data." Gee, that sounds a lot like a holodeck. We've come a long way from using Wii Sensor Bars[0] for position tracking. [0] https://www.youtube.com/watch?v=LC_KKxAuLQw

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's thing was that it was really cheap. But yes, damn close to a holodeck. But you can't see depth in this setup, right?

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not

And that's certainly not the goal with this. Something along those lines has been around for a while (https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ...). This system seems specifically targeted for solving challenges for film production, as it probably should be.

I am pretty impressed that real time rendering has gotten good enough to use for these purposes. I certainly wouldn't have expected that those backgrounds in the show were coming out of a video game engine.

Re: Real-time, in-camera background compositing in The Mandalorian

#12

Post by modeless »

cbhl wrote: post #2

"The solution was ... a dynamic, real-time, photo-real background played back on a massive LED video wall and ceiling ... rendered with correct camera positional data." Gee, that sounds a lot like a holodeck. We've come a long way from using Wii Sensor Bars[0] for position tracking. [0] https://www.youtube.com/watch?v=LC_KKxAuLQw

The "holodeck" version of this is called a CAVE and the first one was built in 1992: https://www.youtube.com/watch?v=aKL0urEdtPU https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ...

Re: Real-time, in-camera background compositing in The Mandalorian

#13

Post by jahlove »

oseibonsu wrote: post #3

Here is the Unreal Engine tech they are using: https://www.unrealengine.com/en-US/spotlights/unreal-engine-... . This is a video of it in action: https://www.youtube.com/watch?v=bErPsq5kPzE&feature=emb_logo .

Here's a video of it in action on The Mandalorian set:

https://www.youtube.com/watch?v=gUnxzVOs3rk

Re: Real-time, in-camera background compositing in The Mandalorian

#15

Post by RaptorJ »

nocut12 wrote: post #11
ragebol wrote: post #4

Earlier quoted context omitted.

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's thing was that it was really cheap. But yes, damn close to a holodeck. But you can't see depth in this setup, right?

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while ( https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ... ). This system seems specifically targeted for solving challenges for film production, as it probably should be. I…

From the video posted upthread, around 3:40 it looks like they're doing just that: https://youtu.be/gUnxzVOs3rk?t=220

Re: Real-time, in-camera background compositing in The Mandalorian

#16

Post by miohtama »

nocut12 wrote: post #11
ragebol wrote: post #4

Earlier quoted context omitted.

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's thing was that it was really cheap. But yes, damn close to a holodeck. But you can't see depth in this setup, right?

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while ( https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ... ). This system seems specifically targeted for solving challenges for film production, as it probably should be. I…

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

Re: Real-time, in-camera background compositing in The Mandalorian

#17

Post by KineticLensman »

oseibonsu wrote: post #3

Here is the Unreal Engine tech they are using: https://www.unrealengine.com/en-US/spotlights/unreal-engine-... . This is a video of it in action: https://www.youtube.com/watch?v=bErPsq5kPzE&feature=emb_logo .

Unreal Engine is also used by the BBC to create virtual studios for football punditry programmes. This uses a simpler green screen technology, but it demonstrates how Epic are moving away from their gaming roots.

Re: Real-time, in-camera background compositing in The Mandalorian

#18

Post by DonHopkins »

"Once Upon a Time" (2011-2018, with Robert Carlyle as Rumplestiltskin!) was shot on virtual chroma-keyed sets with real time integrated pipeline tools to preview how it would look.

https://en.wikipedia.org/wiki/Once_Upon_a_Time_(TV_series)

The tech behind Once Upon a Time’s Frozen adventures

https://www.fxguide.com/fxfeatured/the-tech-behind-once-upon...

Once Upon a Time” TV Series VFX Breakdown

https://web.archive.org/web/20180623020817/http://www.animat...

Post reply on HN