Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real-time, in-camera background compositing in The Mandalorian

ascmag.com

Post reply on HN
Print view View original

81–90 of 126 posts

Re: Real-time, in-camera background compositing in The Mandalorian

#81

Post by devindotcom »

mitchelhall wrote: post #50

Hi all, really cool that you have taken an interest in this project, a lot of your comments below are very insightful and interesting. I work on the team that deploys this tech on set. We focus on how the video signals get from the video output servers to the LED surfaces, coordinating how we deal with tracking methods, photogrammetry, signal delivery, operator controls and the infrastructure that supports all of thi…

Would love to talk to you! Are you at ILM or a partner like Fuse or ROE? You can contact me at devin at techcrunch dot com, I'm working on more pieces on this tech.

Re: Real-time, in-camera background compositing in The Mandalorian

#82

Post by Cthulhu_ »

miohtama wrote: post #16
nocut12 wrote: post #11

Earlier quoted context omitted.

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while ( https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ... ). This system seems specifically targeted for solving challenges for film production, as it probably should be. I…

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

One solution to this would be to put the camera on a fixture that replays the same motions every time, so they can do a 'dry run' and correct the rendering. (Putting a camera on a fixture is not new, IIRC they did it in Back to the Future - https://www.youtube.com/watch?v=AtPA6nIBs5g is a really good yet succinct documentary on it)

Re: Real-time, in-camera background compositing in The Mandalorian

#83

Post by gmueckl »

Aeolun wrote: post #42
miohtama wrote: post #16

Earlier quoted context omitted.

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

I don’t fully get this. They could just employ different computers to render different parts of their cave. It seems more like a cost savings thing than a technical limitation. And I’m not sure why you’d skimp on a few PC’s if you’re already building a humongous led wall, so maybe there’s something I’m missing.

The part you are missing is the insane complexity involved in keeping perfect frame sync with a low latency across GPUs and machines, especialliy when some final compositing of partial outputs is involved. The stuff sounds simple on paper and it looks like you can just go buy the tech, unpack it and switch it on. The reality is nothing like that. The off the shelf tech is fiddly and barely stable because it is always a low priority feature added with the least possible effort.

Re: Real-time, in-camera background compositing in The Mandalorian

#84

Post by severak_cz »

Funny that this is practically the same concept as shooting in atelier with exterior background painted on walls as it was done in old movies. The progress is only in technology - now it's created by game engine and projected to giant LEDs, back then in 1930s it was done by hand by painters.

Re: Real-time, in-camera background compositing in The Mandalorian

#86

Post by andybak »

ragebol wrote: post #76
whatshisface wrote: post #6

Earlier quoted context omitted.

They have polarized 3D screens, and with head tracking, you have it.

Was thinking the same, but that'll also work for just 1 PoV

You could multiplex more images if the glasses were synced and actively polarized. Every person gets a timeslice of the total image.

Re: Real-time, in-camera background compositing in The Mandalorian

#87

Post by GonzaloQuero »

Aeolun wrote: post #42
miohtama wrote: post #16

Earlier quoted context omitted.

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

I don’t fully get this. They could just employ different computers to render different parts of their cave. It seems more like a cost savings thing than a technical limitation. And I’m not sure why you’d skimp on a few PC’s if you’re already building a humongous led wall, so maybe there’s something I’m missing.

I worked in a somewhat similar project in 2015, though not as complex as this, to build background videos for DotParty, using UE4 for panoramas and then stitching them. One of the hugest issues we found was that, because of this being a game engine, a lot of things are not deterministic, so if we used multiple cards or computers, particles and other environmental effects would not be in sync, and the stitches were glaring.

Re: Real-time, in-camera background compositing in The Mandalorian

#88

Post by [deleted] »

Cthulhu_ wrote: post #82
miohtama wrote: post #16

Earlier quoted context omitted.

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

One solution to this would be to put the camera on a fixture that replays the same motions every time, so they can do a 'dry run' and correct the rendering. (Putting a camera on a fixture is not new, IIRC they did it in Back to the Future - https://www.youtube.com/watch?v=AtPA6nIBs5g is a really good yet succinct documentary on it)

[deleted]

Re: Real-time, in-camera background compositing in The Mandalorian

#89

Post by dredmorbius »

tobr wrote: post #73
devindotcom wrote: post #5

This is super interesting stuff and I've been following it for some time. I just wrote it up with a bit more context: https://techcrunch.com/2020/02/20/how-the-mandalorian-and-il... It's not just ILM and Disney either, this is going to be everywhere . It's complex to set up and run in some ways but the benefits are enormous for pretty much everyone involved. I doubt there will be a major TV or film production that do…

Is there any way to read your article if I can’t figure out how to navigate through the legalese and dark patterns of Verizon’s privacy extortion wall?

https://outline.com/https://techcrunch.com/2020/02/20/how-th...

Re: Real-time, in-camera background compositing in The Mandalorian

#90

Post by frostburg »

ragebol wrote: post #74
babypuncher wrote: post #27

Earlier quoted context omitted.

It won't trick anyone with stereo vision, but the perspective correction provides all the depth cues needed to create a convincing 3D visual for pirates and non-stereo cameras.

pirates? What arrrr you talking about?

Eyepatches.
Post reply on HN