Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pixar's Render Farm

twitter.com

Post reply on HN
Print view View original

101–110 of 323 posts

Re: Pixar's Render Farm

#101

Post by ArtWomb »

berkut wrote: post #54
mmcconnell1618 wrote: post #40

Can anyone comment on why Pixar uses standard CPU for processing instead of custom hardware or GPU? I'm wondering why they haven't invested in FPGA or completely custom silicon that speeds up common operations by an order of magnitude. Is each show that different that no common operations are targets for hardware optimization?

Because the expense is not really worth it - even GPU rendering (while around 3/4 x faster than CPU rendering) is memory constrained compared to CPU rendering, and as soon as you try and go out-of-core on the GPU, you're back at CPU speeds, so there's usually no point doing GPU rendering for entire scenes (which can take > 48 GB of RAM for all geometry, accel structures, textures, etc) given the often large memory re…

Nice to have an industry insider perspective on here ;)

Can you speak to any competitive advantages a vfx-centric gpu cloud provider may have over commodity AWS? Even the RenderMan XPU looks to be OSL / Intel AVX-512 SIMD based. Thanks!

Supercharging Pixar's RenderMan XPU™ with Intel® AVX-512

https://www.youtube.com/watch?v=-WqrP50nvN4

Re: Pixar's Render Farm

#103

Post by _3r2w »

nightfly wrote: post #81
aprdm wrote: post #68

Earlier quoted context omitted.

FPGA is really expensive for the scale of a modern studio render farm, we're talking around 40~100k cores per datacenter. Because 40~100k cores isn't Google scale either it also doesn't seem to make sense to invest in custom silicon. There's a huge I/O bottleneck as well as you're reading huge textures (I've seen textures as big as 1 TB) and writing constantly to disk the result of the renderer. Other than that, most…

I'm assuming these 1TiB textures are procedural generated or composites? Where do this large of textures come up?

1 terabyte sounds like an outlier, but typically texture maps are used as inputs to shading calculations. So it's not uncommon for hero assets in large-scale VFX movies to have more than 10 different sets of texture files that represent different portions of a shading model. For large assets, it may take more than fifty 4K-16K images to adequately cover the entire model such that if you were to render it from any angle, you wouldn't see the pixelation. And these textures are often stored as mipmapped 16 bit images so the renderer can choose the most optimal resolution at rendertime.

So that can easily end up being several hundred gigabytes of source image data. At rendertime, only the textures that are needed to render what's visible in the camera are loaded into memory and utilized, which typically ends up being a fraction of the source data.

Large scale terrains and environments typically make more use of procedural textures, and they may be cached temporarily in memory while the rendering process happens to speed up calculations

Re: Pixar's Render Farm

#104

Post by dahart »

mmcconnell1618 wrote: post #40

Can anyone comment on why Pixar uses standard CPU for processing instead of custom hardware or GPU? I'm wondering why they haven't invested in FPGA or completely custom silicon that speeds up common operations by an order of magnitude. Is each show that different that no common operations are targets for hardware optimization?

> Can anyone comment on why Pixar uses standard CPU for processing instead of custom hardware or GPU?

A GPU enabled version of RenderMan is just coming out now. I imagine their farm usage after this could change.

https://gfxspeak.com/2020/09/11/animation-studios-renderman/

I’m purely speculating, but I think the main reason they haven’t been using GPUs until now is that RenderMan is very full featured, extremely scalable on CPUs, has a lot of legacy features, and it takes a metric ton of engineering to port and re-architect well established CPU based software over to the GPU.

Re: Pixar's Render Farm

#105

Post by dagmx »

banana_giraffe wrote: post #95

One of the things they mentioned briefly in a little documentary on the making of Soul is that all of the animators work on fairly dumb terminals connected to a back end instance. I can appreciate that working well when people are in the office, but I'm amazed that worked out for them when people moved to work from home. I have trouble getting some of my engineers to have a stable connection stable enough for VS Code…

A lot of studios use thin client/ PCoiP boxes from teradici etc..

They're pretty great overall and the bandwidth requirements aren't crazy high but it does max out your data usage if you're capped pretty quickly. The faster you can be, the better the experience.

Some studios like Imageworks don't even have the backend data center in the same location. So the thin clients connect to a center in Washington state when the studios are in LA and Vancouver.

Re: Pixar's Render Farm

#106

Post by NikolaNovak »

naikrovek wrote: post #70
xtracto wrote: post #65

Earlier quoted context omitted.

Twitter was designed to have 280 characters max per message. This means that for this kind of long format text, the amount of Signal-to-Noise ratio of having a large number of "tweets" is pretty low. The amount of stuff your brain has to filter in the form of user name, user tweet handle, additional tagged handlers, UI menus, UI buttons for replying, retweeting, liking, etc on every single code snippet makes your bra…

You all are perfect delicate flowers that need things to be just right in order to use them, then? Is that what you're saying? Because that's what I'm getting from you.

I mean,yes? :-)

If I'm going to use something, it should be intuitive and usable. It should be fit for its purpose, especially with a myriad of single and multi purpose tools available to all. This doesn't feel like something I should justify too hard :-)

Twitter is not a necessity of life. I don't have to use it. They want me to use it and if so they can/should make it usable.

Its paradigm and user interface don't work for me personally (and particularly when people try to fit an article into something explicitly designed for a single sentence - it feels like a misuse of a tool,like hammering a screw) so I don't use it. And that's ok!

I don't feel they are morally obligated to make it usable by me. It's a private platform and they can do as they please.

But my wife is a store manager and taught me that "feedback is a gift" - if a customer will leave the store and never come back,she'd rather know why, rather then remain ignorant.

She may or may not choose to address it but being aware and informed is better than being ignorant of the reasons.

So at the end of it, rather than down vote, let me ask what is the actual crux of your argument? People shouldn't be discriminate? They should use optional things they dislike? They shouldn't share their preferences and feedback? Twitter is a great tool for long format essays? Or something we all may be missing?

Re: Pixar's Render Farm

#107

Post by dahart »

aprdm wrote: post #68
mmcconnell1618 wrote: post #40

Can anyone comment on why Pixar uses standard CPU for processing instead of custom hardware or GPU? I'm wondering why they haven't invested in FPGA or completely custom silicon that speeds up common operations by an order of magnitude. Is each show that different that no common operations are targets for hardware optimization?

FPGA is really expensive for the scale of a modern studio render farm, we're talking around 40~100k cores per datacenter. Because 40~100k cores isn't Google scale either it also doesn't seem to make sense to invest in custom silicon. There's a huge I/O bottleneck as well as you're reading huge textures (I've seen textures as big as 1 TB) and writing constantly to disk the result of the renderer. Other than that, most…

> There are studios using GPUs for some workflows but most of it is CPUs.

This is probably true today, but leaves the wrong impression IMHO. The clear trend is moving toward GPUs, and surprisingly quickly. Maya & Houdini have release GPU simulators and renderers. RenderMan is releasing a GPU renderer this year. Most other third party renderers have already gone or are moving to the GPU for path tracing - Arnold, Vray, Redshift, Clarisse, etc., etc.

Re: Pixar's Render Farm

#108

Post by turminal »

naikrovek wrote: post #70
xtracto wrote: post #65

Earlier quoted context omitted.

Twitter was designed to have 280 characters max per message. This means that for this kind of long format text, the amount of Signal-to-Noise ratio of having a large number of "tweets" is pretty low. The amount of stuff your brain has to filter in the form of user name, user tweet handle, additional tagged handlers, UI menus, UI buttons for replying, retweeting, liking, etc on every single code snippet makes your bra…

You all are perfect delicate flowers that need things to be just right in order to use them, then? Is that what you're saying? Because that's what I'm getting from you.

I for one am saying Twitter has many, many people working on UX and the web interface is still terrible. Hardly any interface made for such a broad spectrum of people gets it just right for all its users, but Twitter is doing an exceptionally bad job at it.

Re: Pixar's Render Farm

#109

Post by mac01021 »

lmilcin wrote: post #59
nikolay wrote: post #47

Earlier quoted context omitted.

It's very painful to follow a conversation on Twitter. I'm not sure why they think the way they've done things makes sense.

It was never supposed to support conversation in the first place. People were supposed to shoot short, simple, single messages and other people maybe react to this with their own short, single messages.

That sounds a lot like a conversation to me.

Re: Pixar's Render Farm

#110

Post by lattalayta »

ArtWomb wrote: post #101
berkut wrote: post #54

Earlier quoted context omitted.

Because the expense is not really worth it - even GPU rendering (while around 3/4 x faster than CPU rendering) is memory constrained compared to CPU rendering, and as soon as you try and go out-of-core on the GPU, you're back at CPU speeds, so there's usually no point doing GPU rendering for entire scenes (which can take > 48 GB of RAM for all geometry, accel structures, textures, etc) given the often large memory re…

Nice to have an industry insider perspective on here ;) Can you speak to any competitive advantages a vfx-centric gpu cloud provider may have over commodity AWS? Even the RenderMan XPU looks to be OSL / Intel AVX-512 SIMD based. Thanks! Supercharging Pixar's RenderMan XPU™ with Intel® AVX-512 https://www.youtube.com/watch?v=-WqrP50nvN4

One potential difference is that the input data required to render a single frame of a high end animated or VFX movie might be several hundred gigabytes (even terabytes for heavy water simulations or hair) - caches, textures, geometry, animation & simulation data, scene description. Often times a VFX centric cloud provider will have some robust system in place for uploading and caching out data across the many nodes that need it. (https://www.microsoft.com/en-us/avere)

And GPU rendering has been gaining momentum over the past few years, but the biggest bottleneck until recently was availabe VRAM. Big budget VFX scenes can often take 40-120 GB of memory to keep everything accessible during the raytrace process, and unless a renderer supports out-of-core memory access, then the speed up you may have gained from the GPU gets thrown out the window from swapping data

Post reply on HN