Pixar's Render Farm • Hacker News Board

Re: Pixar's Render Farm

#291

Post by forelle2 » Sun, Jan 03, 2021, 6:59 PM UTC

CyberDildonics wrote: post #287

forelle2 wrote: post #254

Earlier quoted context omitted.

The largest texture sets are heading towards 1TB in size, or at least they were when I was last involved in production support. I saw Mari projects north of 650gb and that was 5 years ago. Disclaimer : I wrote Mari, the vfx industry standard painting system. Note though these are not single 1TB textures, they’re multiple sets of textures, plus all of the layers that constitute them. Some large robots In particular ha…

I think we both realize that it's a bit silly to have so much data in textures that you have 100x the pixel data of a 5 second shot at 4k with 32 bit float rgb. 650GB of textures would mean that even with 10gb ethernet (which I'm not sure is common yet) you would wait at least 12 minutes just for the textures to get to the computer before rendering could start and rendering 100 frames at a time would mean 100GB/s fro…

Texture caching in modern renders tends to be on demand and paged so it is very unlikely the full texture set is ever pulled from the filers.

Over texturing like this can be a good decision depending on the production. Asset creation often starts a long time before shots or cameras are locked down.

If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4K.

Taking an asset off final to ‘Upres’ it for a can be a pain in the ass and more costly than just detailing it up in the first place.

In isolation it’s a insane amount of detail and given perfect production planning it is normally not needed, but until directors lock down the scripts and shots it can be the simplest option.

Re: Pixar's Render Farm

#292

Post by roboyoshi » Sun, Jan 03, 2021, 7:16 PM UTC

jedimastert wrote: post #100

When I was a little younger, I was looking at 3D graphics as a career path, and I knew from the very beginning that if I were to do it, I would work towards Pixar. I've always admired everything they've done, both from an artistic and technical standpoint, and how well they've meshed those two worlds in an incredible and beautiful manner.

Same here. I'm older now and the dream is lost a bit, but in the beginning it really drove me towards some good and kowledgeable people. I worked in Film (and Animation), but once I saw the working conditions (low pay, long hours, difficult to find work) I moved to SWE.. I still rarely do some work on the side for VFX/Animation.. and I think that is the way I keep it so I don't loose the "fun" in all of it.

Re: Pixar's Render Farm

#293

Post by jason_slack » Sun, Jan 03, 2021, 7:40 PM UTC

berkut wrote: post #169

KaiserPro wrote: post #150

Earlier quoted context omitted.

> How do they provision it ex VFX sysadmin here. I'm not sure if they use their own scheduler or not. IF they do, they use tractor(might be tractor 2 now), which looks after putting the processes in the right places. Think K8s, but actually easy to use, well documented and reliable. (just not distributed, but then it scales way higher and is nowhere near as chatty) They would have a whole bunch of machines, some old…

Rumour in the industry is that Pixar don't use Tractor themselves, and have a custom solution in Emeryville :)

Do you have a link to Tractor to let folks check it out?

Re: Pixar's Render Farm

#294

Post by CyberDildonics » Sun, Jan 03, 2021, 8:58 PM UTC

forelle2 wrote: post #291

CyberDildonics wrote: post #287

Earlier quoted context omitted.

I think we both realize that it's a bit silly to have so much data in textures that you have 100x the pixel data of a 5 second shot at 4k with 32 bit float rgb. 650GB of textures would mean that even with 10gb ethernet (which I'm not sure is common yet) you would wait at least 12 minutes just for the textures to get to the computer before rendering could start and rendering 100 frames at a time would mean 100GB/s fro…

Texture caching in modern renders tends to be on demand and paged so it is very unlikely the full texture set is ever pulled from the filers. Over texturing like this can be a good decision depending on the production. Asset creation often starts a long time before shots or cameras are locked down. If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4…

> Texture caching in modern renders tends to be on demand and paged so it is very unlikely the full texture set is ever pulled from the filers.

This was easier to rely on in the days before ray tracing, when texture filtering was consistent because everything was from the camera. Ray differentials from incoherent rays aren't quite as forgiving.

> If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4K.

4k textures for large parts of the asset in the UV layout can be an acceptable amount of overkill. That's not the same as putting 65,000 4k textures on something because each little part is given its own 4k texture. I know that you know this, but I'm not sure why you would conflate those two things.

> Taking an asset off final to ‘Upres’ it for a can be a pain in the ass and more costly than just detailing it up in the first place

It is very rare that specific textures need to be redone like that and it is not a big deal.

650GB of textures for one asset drags everything from iterations to final renders to disk usage to disk activity to network usage down for every shot in a completely unnecessary way. There isn't a fine line between these things, there is a giant gap between that much excessive texture resolution and needing to upres some piece because it gets close to the camera.

> Asset creation often starts a long time before shots or cameras are locked down.

This is actually fairly rare.

> In isolation it’s a insane amount of detail and given perfect production planning it is normally not needed, but until directors lock down the scripts and shots it can be the simplest option.

That's rarely how the time line fits together. It's irrelevant though, because there is no world where 65,000 4k textures on a single asset makes sense. It's multiple orders of magnitude out of bounds of reality.

I am glad that you have that insane amount of scalability as a focus since you are making tools that people rely on heavily, and I wish way more people on the tools end thought like this. Still, it is about 1000x what would set off red flags in my mind.

I apologize on behalf of whoever told you that was necessary, because they need to learn how to work within reasonable resources (which is not difficult given modern computers), no matter what project or organization they are attached to.

Re: Pixar's Render Farm

#295

Post by corysama » Sun, Jan 03, 2021, 10:59 PM UTC

greggman3 wrote: post #253

corysama wrote: post #118

Earlier quoted context omitted.

Not an ILMer, but I was at LucasArts over a decade ago. Back then, us silly gamedevs would argue with ILM that they needed to transition from CPU to GPU based rendering. They always pushed back that their bottleneck was I/O for the massive texture sets their scenes through around. At the time RenderMan was still mostly rasterization based. Transitioning that multi-decade code and hardware tradition over to GPU would…

AFAIU the issue with GPU rendering is generally you have to design the assets to be GPU friendly. So while you can get a huge speed up at rendering time you get a huge slowdown creating the assets in the first place because you have new issues. Using normal maps and displacement maps instead of millions of polygons. Keeping textures to the minimal size that will get the job done, etc... Is any of that true?

Not for the ILM use-case. I expect they would stick to finely-tessellated geometry. The challenge would be moving all of that data across the PCI bus in and out of the relatively limited DRAM on the GPU. It would require a very intelligent streaming solution. Similar to the one they already have to stream resources from storage to the CPU RAM of various systems.

Re: Pixar's Render Farm

#296

Post by [deleted] » Mon, Jan 04, 2021, 1:33 AM UTC

naikrovek wrote: post #74

rconti wrote: post #64

Earlier quoted context omitted.

It's very unintuitive that when you open a Thread, the "back" arrow means "back to something else" and you have to scroll up above "line 0" to see the context of the thing being replied to. I forget this every single time I open a tweet thread and try to figure out the context. Once you scroll up, it sort of makes sense -- each tweet with a line connecting user icons but then suddenly the expanded tweet thread has th…

I am not saying it is intuitive. I'm saying it's usable. I'm saying that complaining about it makes people look like they think they're royalty who need everything just so or their whole day is ruined... And now they can't tell the butler to take Poopsie for a walk because they're so shaken by the experience.

[deleted]

Re: Pixar's Render Farm

#297

Post by nikolay » Mon, Jan 04, 2021, 2:14 AM UTC

ganafagol wrote: post #250

nikolay wrote: post #205

Earlier quoted context omitted.

If there's will, there's a way. Remember how long they've tried to convince themselves that 140 characters is more than enough. Nowadays, even phones display conversations better and combine or split text messages hiding the 160 characters limit from the end user and allow longer than 280 characters texts. Twitter has some nice ideas, but it's mostly a huge missed opportunity.

The character limit was central to making Twitter what it is. Without that limit you'd just have ended up with walls of text and nobody would have been interested. This is a great example of limitations forcing creativity. The character limit is the defining feature of Twitter.

Maybe for you. I'm not saying having no limit, but a number of words limit is better than a number of characters. They started to revert their _defining feature_ but not counting characters in URLs, etc.

Re: Pixar's Render Farm

#298

Post by jlgaddis » Mon, Jan 04, 2021, 2:38 AM UTC

CamJN wrote: post #233

jlgaddis wrote: post #229

Earlier quoted context omitted.

If the very light gray color of your comment is any indication, folks here didn't like what you said. I certainly don't think that HN should ban you because of it, though.

[flagged]

You shouldn't be so quick to jump to conclusions! To be clear, I'm about as far from "right wing" as one can get. I wasn't aware that standing up for the First Amendment rights of others was a partisan issue, though.

Now, I realize that -- in today's divided society -- I'm supposed to be completely on board with silencing those who hold different opinions or disagree with "our" beliefs ("canceling", I think it's called?) but, well, it just don't work like that.

See, I'm from the rural midwest (a self-proclaimed "country boy"), I drive a big 4WD truck, I ride a loud ass Harley-Davidson motorcycle, and there's three or four times as many guns in this house are there are people. My hometown, my family, my friends, and my acquaintances are all overwhelmingly Republicans -- including some of the people that I love and care for the most in this world -- yet, somehow, I've been a Democrat for my entire adult life.

Contrary to what some folks in "my" party seem to think, however, EVERYONE (still) has the right to their own beliefs and opinions -- and to express them -- no matter how ignorant, ill-informed, asinine, outright stupid, or batshit crazy they may be!

We have a saying around here in "my neck of the woods": I may not agree with what you say but I will fight for and defend to my death your absolute right to say it.

For the record, the only time I've even come close to attempting to "silence" someone or prevent them from exercising their rights has been at funerals -- as a (proud) member of the Patriot Guard Riders.

Re: Pixar's Render Farm

#299

Post by forelle2 » Mon, Jan 04, 2021, 2:56 AM UTC

CyberDildonics wrote: post #294

forelle2 wrote: post #291

Earlier quoted context omitted.

Texture caching in modern renders tends to be on demand and paged so it is very unlikely the full texture set is ever pulled from the filers. Over texturing like this can be a good decision depending on the production. Asset creation often starts a long time before shots or cameras are locked down. If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4…

> Texture caching in modern renders tends to be on demand and paged so it is very unlikely the full texture set is ever pulled from the filers. This was easier to rely on in the days before ray tracing, when texture filtering was consistent because everything was from the camera. Ray differentials from incoherent rays aren't quite as forgiving. > If you don’t know how an asset is to be used it makes sense to texture…

Mari was designed in production at Weta, based off the lessons learned from, well, everything that Weta does.

Take for example, a large hero asset like King Kong.

Kong look development started many months before a script was locked down. Kong is 60ft tall, our leading lady is 5’2”.

We think we need shots where she’ll be standing in Kong’s hands, feet, be lifted up to his face, nose etc.

So we need fingers prints that will stand up at 4K renders, tear ducts, pores on the inside on the nose, etc etc but we don’t know. All of which will have to match shot plates in detail.

We could address each of these as the shots turn up and tell the director (who owns the company) he needs to wait a few days for his new shot, or you can break Kong into 500 patches and create a texture for each of the diffuse, 3 spec, 3 subsurface, 4 bump, dirt, blood, dust, scratch, fur, flow etc etc inputs to our shaders.

Let’s says we have 500 UDIM patches for Kong so we can sit our leading lady on the finger tips, and 20 channels to drive our shaders and effects systems.

When working the artist uses 6 paint layers for each channel ( 6 is a massive underestimate for most interesting texture work).

So we have 500 patches * 20 channels * 6 layers which gives us 60k images. Not all of these will need be at 4K however.

For Kong replace any hero asset where shots will be more placed “in and on” the asset rather than “at”. Heli carriers, oil rigs, elven great halls, space ships, giant robots.... The line between asset and environment is blurred at that point and maybe think “set” rather than “asset”

Re: Pixar's Render Farm

#300

Post by rperez333 » Mon, Jan 04, 2021, 3:45 AM UTC

dagmx wrote: post #164

dodobirdlord wrote: post #147

Earlier quoted context omitted.

That seems extremely unlikely. The Renderman software they use has no issues rendering at 4k.

It's not really that unlikely. Most films render at 2k DCI. It's not so much that the software and hardware can't render higher, it's just diminishing gains for the increased render time. Most 4k films till very recently actually have the digital elements at 2k DCI-ish resolutions and are upscaled. I can't speak to wether soul is rendered at 2k or 4k, but it wouldn't be surprising if it was 2k upscaled.

That's more accurate, Soul was probably rendered at 2k, as most of their movies.

If a shot doesn't look good enough when upscaled, they re-render in 4k.

Actually, they used to upscale most of the movies using Nuke, but recently, started using deep learning for that. At Siggraph 2020, they gave a talk on this topic: https://s2020.siggraph.org/presentation/?id=gensub_443&sess=...