Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real-time, in-camera background compositing in The Mandalorian

ascmag.com

Post reply on HN
Print view View original

111–120 of 126 posts

Re: Real-time, in-camera background compositing in The Mandalorian

#111

Post by canada_dry »

marcan_42 wrote: post #92
kragen wrote: post #61

Earlier quoted context omitted.

Thanks! This work is really inspiring! Are these OLED screens, matrices of separate LEDs of the usual InGaAs and GaN type, or LCDs backlit with LEDs? The 2.84 mm pixel pitch makes it sound like it's separate inorganic LEDs. Are there times short of sunshine where you need more directionality to the lighting than the screens can provide? Because the screens can emit from any direction but not toward any direction, bei…

Discrete LED screens go all the way down to 2mm or even 1.9mm pitch (I have a few 2mm tiles at home). These are certainly discrete LED panels, not OLEDs. I'm not aware of OLEDs being used in "wall" applications like this. When you hear LED walls, think millions of discrete LEDs mounted on PCBs (and thank China for making this low cost enough to be viable!)

> low cost enough

The article doesn't mention cost, but I'm expecting it was in the $5-10M range. Anyone have actual figures for this type of hi-res, hi-lumen hi-refresh LED screen?

Re: Real-time, in-camera background compositing in The Mandalorian

#112

Post by kragen »

marcan_42 wrote: post #92
kragen wrote: post #61

Earlier quoted context omitted.

Thanks! This work is really inspiring! Are these OLED screens, matrices of separate LEDs of the usual InGaAs and GaN type, or LCDs backlit with LEDs? The 2.84 mm pixel pitch makes it sound like it's separate inorganic LEDs. Are there times short of sunshine where you need more directionality to the lighting than the screens can provide? Because the screens can emit from any direction but not toward any direction, bei…

Discrete LED screens go all the way down to 2mm or even 1.9mm pitch (I have a few 2mm tiles at home). These are certainly discrete LED panels, not OLEDs. I'm not aware of OLEDs being used in "wall" applications like this. When you hear LED walls, think millions of discrete LEDs mounted on PCBs (and thank China for making this low cost enough to be viable!)

In this case, 106 168 320 discrete LEDs, if we assume RGB and believe the 12288×2160 + 4096×2160 figure in the article; or 35 389 440 discrete LEDs if we count each (presumably RGB) pixel as a single unit, or if they're using a Bayer-pattern screen like PenTile AMOLEDs.

Re: Real-time, in-camera background compositing in The Mandalorian

#113

Post by russellbeattie »

jpmattia wrote: post #104

I've had a peripheral interest in virtual sets and real-time compositing by way of a colleague from grad school. A quick visual summary of this tech: http://www.lightcrafttech.com/portfolio/beauty-beast/ This video was from a pilot several years ago, and it didn't make it to air, but it was visually wonderful.

With F. Murray Abraham as well! Nice. He's looked the same since 1984's Amadeus. Crazy!

Re: Real-time, in-camera background compositing in The Mandalorian

#114

Post by estebank »

ChuckNorris89 wrote: post #77
rebuilder wrote: post #7

Earlier quoted context omitted.

Mocap systems haven't really been able to produce deliverable results without human intervention very long. I'd argue they're still not there, some filtering and cleanup of the data is usually required. A lot of VFX is still about throwing human labour at problems. Edit: I should note I'm talking about motion capture for characters etc. Capturing the motion of a rigid object like a camera in a controlled environment…

You're right. The mocap data for the actor who played Thanos in Avengers had to be post processed by artist by hand since the motion of a regular man doesn't correspond to the motion of a larger and heavier Titan. I guess in a few years all that could be automated with ML.

You don't need ML to solve that problem, you can use inverse kinematics:

https://youtu.be/KLjTU0yKS00

Re: Real-time, in-camera background compositing in The Mandalorian

#115

Post by overcast »

mercer wrote: post #110
overcast wrote: post #30

Earlier quoted context omitted.

Speaking of the Wii, does anyone else wish that motion controls could just be removed from its existing library? Not only has it just a major annoyance in most games, but its basically locked into that hardware now.

Metroid Prime is one of the few games I've played all the way through multiple times. When it became increasingly difficult to do so on a GameCube, I relented and tried the ii U version. Much as I've tried,I can't get comfortable with the controls. Thankfully I finally have a laptop that can properly emulate the GC version and allow me to use the GC controller that I dusted off. But I can't wait for a GC-inspired ver…

I've been procrastinating playing Metroid Prime 3 because I don't want to waggle around the whole time. Pumped for the endlessly delayed Switch version though, I've had it preordered FOREVER.

Re: Real-time, in-camera background compositing in The Mandalorian

#116

Post by estebank »

severak_cz wrote: post #84

Funny that this is practically the same concept as shooting in atelier with exterior background painted on walls as it was done in old movies. The progress is only in technology - now it's created by game engine and projected to giant LEDs, back then in 1930s it was done by hand by painters.

I think the innovation is the perspective correction of the background depending on the camera. That could have been accomplished with rear projection in film if it had been necessary by having the camera follow a preset path, but I don't think even BTTF attempted that.

Re: Real-time, in-camera background compositing in The Mandalorian

#117

Post by estebank »

janekm wrote: post #101
treblig wrote: post #59

"Postproduction was mostly refining creative choices that we were not able to finalize on the set in a way that we deemed photo-real." Does anyone know how they were able to swap out the in-camera version of the background originally shown on the LED wall with something more convincing later? Seems like it'd be tough since it's not a green screen!

While currently they are using "green screen" in those instances, given that the camera positions are already being tracked, and the image displayed on the screens is known, it would be possible to re-render what image the camera should have seen if the foreground elements hadn't been present and use the difference with the recorded image as a mask for further post-processing. (which would be very cool as it would al…

It seems to me that making the system be able to perform masking itself (instead of projecting a green screen) would reduce the ability to completely replace the scene easily, but the reflections already give you that problem. The advantage you would have is that the real elements in the scene would no longer need any feathering around the edges causing them to look hazy, because now you can take a few pixels of the projected background as a transition to the higher resolution render.

Re: Real-time, in-camera background compositing in The Mandalorian

#118

Post by devindotcom »

tobr wrote: post #96
dredmorbius wrote: post #89

Earlier quoted context omitted.

https://outline.com/https://techcrunch.com/2020/02/20/how-th...

Some of the behind the scenes photos are really uncanny. Your eyes tell you you’re looking at a photo of an outdoor scene, but the illusion of reality breaks down in subtle ways at the edges of the image.

Interesting, right? I imagine it may actually be kind of distracting for the actors, though it has to be better than working in a giant bright green cave. You should watch the behind the scenes video, seeing it in motion is really cool.

Re: Real-time, in-camera background compositing in The Mandalorian

#119

Post by lsaferite »

babypuncher wrote: post #27
ragebol wrote: post #4

Earlier quoted context omitted.

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's thing was that it was really cheap. But yes, damn close to a holodeck. But you can't see depth in this setup, right?

It won't trick anyone with stereo vision, but the perspective correction provides all the depth cues needed to create a convincing 3D visual for pirates and non-stereo cameras.

Interestingly I would imagine pulling the non-CG part out of the frame would be possible as they have the ability to generate the exact image without the real-world aspects. Basically a virtual green-screen. Combine that with a stereo camera and the fact that the source CG is actually 3D and you could get very convincing 3D movies I'd think. Yeah, that simplified a lot, but I'd say it's possible. And they still get the benefits of the lighting aspects as well as the immersion for the actors.
Post reply on HN