Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real-time, in-camera background compositing in The Mandalorian

ascmag.com

Post reply on HN
Print view View original

41–50 of 126 posts

Re: Real-time, in-camera background compositing in The Mandalorian

#41

Post by anigbrowl »

lmilcin wrote: post #22

I don't care how "groundbreaking" the graphics pipeline. I watched couple of episodes and I had to force myself to keep watching to, I don't know, give it a chance? I wonder when The Industry figures out the story is more important than the graphics. You don't buy books for beautiful cover and typesetting... at least not most you and not most of the time.

I enjoyed the story and apparently many other people did too. It's fanservice for sure, hence all the callbacks to characters and aliens that had background or very brief appearances in the original movies and left people wanting more. Cheesy, perhaps, but the entire franchise is pineapple pizza in space.

Re: Real-time, in-camera background compositing in The Mandalorian

#42

Post by Aeolun »

miohtama wrote: post #16
nocut12 wrote: post #11

Earlier quoted context omitted.

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while ( https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ... ). This system seems specifically targeted for solving challenges for film production, as it probably should be. I…

They mention they cannot push enough GPU juice to the screens, so they only render the camera focus area in full resolution. Also there is 12 frame lag which prevents moving camera too fast.

I don’t fully get this. They could just employ different computers to render different parts of their cave. It seems more like a cost savings thing than a technical limitation.

And I’m not sure why you’d skimp on a few PC’s if you’re already building a humongous led wall, so maybe there’s something I’m missing.

Re: Real-time, in-camera background compositing in The Mandalorian

#43

Post by petermcneeley »

virtualritz wrote: post #40
petermcneeley wrote: post #25

This technique will produce potentially significant rendering artifacts in the final image. The backdrop is correct only from the position of the camera. A reflection from any surface will not be geometrically correct (as seen by the image from the article) I think that even ambient lighting would contain noticable deformations.

It's much better than the reflection of a green/blue screen or an empty studio with some cameras and people. Glossy surfaces are usually not a problem unless they're (near) perfect mirrors. Even then -- lights are usually what you see in most reflections because they're orders of magnitude brighter than the rest of the set. If reflections are problem with this new technique in certain settings, they would be even mor…

Right but the artifacts here will still show up on glossy surfaces. Here is how games now fix these kinds of artifacts. https://seblagarde.wordpress.com/2012/09/29/image-based-ligh...

However the technology in the article cannot be corrected in this manner.

Re: Real-time, in-camera background compositing in The Mandalorian

#44

Post by tigershark »

Please send me a link of someone that watched the original Star Wars, the later trilogy and finally the last “attempt” and really appreciated it. I even watched “Rogue One” in the biggest screen available around me, with high expectations, and I’m feeling really sad because of that.

Re: Real-time, in-camera background compositing in The Mandalorian

#45

Post by russellbeattie »

Sony had a big demo of this in their CES booth a few weeks ago. They showed the camera moving around the Ghostbuster's Ecto-1 car and the background moving as well. You could see from the overhead screens what the final composite looked like. It was pretty awesome, given it was all set up in a booth at a trade fair. [1]

As expensive as all this is now, I think this is really going to make an impact in lower-budget movies. Not having to fly all over the world or having massive sets to film convincing scenes might be a really good thing.

1. https://www.youtube.com/watch?v=kh2Q9pCxuJw

Re: Real-time, in-camera background compositing in The Mandalorian

#46

Post by marzell »

ghostbrainalpha wrote: post #35
web-cowboy wrote: post #19

So we'll be able to play video game adaptations of the locations in the episodes really easily/soon, right? ;)

That has to be a consideration. But I don't know how much it would really speed up production of a AAA Mandalorian game. Some... maybe a 6 month head start on a 4-5 year game. It would definitely help make the game environments higher quality and be a cost saving to the studio.

It could be really cool to have in-game art using the exact same assets from the actual film. Even whole scene cinematography could be taken from scene data used in a movie.

I can imagine a consumer experience using high-end VR (say an attraction at Disneyland)... take the assets and cinematography from a scene in a movie, digitally recreate everything that WASN'T already digital from the scene, and then you could relive a scene in VR, with the bonus that you can navigate it in realtime and see it from different perspectives. This would be especially adaptable for franchises like Avatar where everything in the film is already composited in 3D.

On a darker note, you could combine all this with things like the Body Transfer Illusion, take some kinda psychedelic and star in your own horror flick where gruesome things are done to your own body in VR. Good times for the whole family!

Re: Real-time, in-camera background compositing in The Mandalorian

#47

Post by cgrealy »

web-cowboy wrote: post #19

So we'll be able to play video game adaptations of the locations in the episodes really easily/soon, right? ;)

I'd say it wouldn't help much. You're building one small scene that's designed to be viewed from a relatively small area or path. I highly doubt they're building anything that doesn't appear on screen. If you were to walk about that scene in Unreal... I'd imagine it's the digital equivalent of a fake old west town.

Re: Real-time, in-camera background compositing in The Mandalorian

#48

Post by cgrealy »

lmilcin wrote: post #22

I don't care how "groundbreaking" the graphics pipeline. I watched couple of episodes and I had to force myself to keep watching to, I don't know, give it a chance? I wonder when The Industry figures out the story is more important than the graphics. You don't buy books for beautiful cover and typesetting... at least not most you and not most of the time.

Whether or not you liked the story is utterly irrelevant to the article.

Re: Real-time, in-camera background compositing in The Mandalorian

#49

Post by sgc »

jahlove wrote: post #13
oseibonsu wrote: post #3

Here is the Unreal Engine tech they are using: https://www.unrealengine.com/en-US/spotlights/unreal-engine-... . This is a video of it in action: https://www.youtube.com/watch?v=bErPsq5kPzE&feature=emb_logo .

Here's a video of it in action on The Mandalorian set: https://www.youtube.com/watch?v=gUnxzVOs3rk

That explains a lot to me. The actors need those visual cues, and it shows through in the final product. It was a great result. I look forward to improved acting as this tech makes its way into other works.

Re: Real-time, in-camera background compositing in The Mandalorian

#50

Post by mitchelhall »

Hi all, really cool that you have taken an interest in this project, a lot of your comments below are very insightful and interesting. I work on the team that deploys this tech on set. We focus on how the video signals get from the video output servers to the LED surfaces, coordinating how we deal with tracking methods, photogrammetry, signal delivery, operator controls and the infrastructure that supports all of this. As noted in some of the comments, the VFX industry solved tracking with mocap suits a long time ago for the post-production process. What we are exploring now is how we can leverage new technology, hardware, and workflows to move some of the post-production processes into the pre-production and production (main/secondary unit photography) stages.

If you are interested in checking out another LED volume setup, my team also worked on First Man last year. This clip shows a little more of how we assemble these LED surfaces as well as a bit of how we use custom integrations to interface with mechanical automation systems. [https://vimeo.com/309382367]

Post reply on HN