Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pixar's Render Farm

twitter.com

Post reply on HN
Print view View original

131–140 of 323 posts

Re: Pixar's Render Farm

#131

Post by mech422 »

naikrovek wrote: post #70
xtracto wrote: post #65

Earlier quoted context omitted.

Twitter was designed to have 280 characters max per message. This means that for this kind of long format text, the amount of Signal-to-Noise ratio of having a large number of "tweets" is pretty low. The amount of stuff your brain has to filter in the form of user name, user tweet handle, additional tagged handlers, UI menus, UI buttons for replying, retweeting, liking, etc on every single code snippet makes your bra…

You all are perfect delicate flowers that need things to be just right in order to use them, then? Is that what you're saying? Because that's what I'm getting from you.

no - I'm a delicate flower that refuses to use that sad excuse of a 'service'...

Plenty of much better, more readable content on the internet without submitting myself to that low quality shit show with a poor ui.

Re: Pixar's Render Farm

#132

Post by lattalayta »

lattalayta wrote: post #110
ArtWomb wrote: post #101

Earlier quoted context omitted.

Nice to have an industry insider perspective on here ;) Can you speak to any competitive advantages a vfx-centric gpu cloud provider may have over commodity AWS? Even the RenderMan XPU looks to be OSL / Intel AVX-512 SIMD based. Thanks! Supercharging Pixar's RenderMan XPU™ with Intel® AVX-512 https://www.youtube.com/watch?v=-WqrP50nvN4

One potential difference is that the input data required to render a single frame of a high end animated or VFX movie might be several hundred gigabytes (even terabytes for heavy water simulations or hair) - caches, textures, geometry, animation & simulation data, scene description. Often times a VFX centric cloud provider will have some robust system in place for uploading and caching out data across the many nodes…

Oh, and also, security. After the Sony hack several years ago, many film studios have severe restrictions on what they'll allow off-site. For upcoming unreleased movies, many studios are overly protective of their IP and want to mitigate the chance of a leak as much as possible. Often times complying with those restrictions and auditing the entire process is enough to make on-site rendering more attractive.

Re: Pixar's Render Farm

#133

Post by berkut »

dahart wrote: post #117
berkut wrote: post #54

Earlier quoted context omitted.

Because the expense is not really worth it - even GPU rendering (while around 3/4 x faster than CPU rendering) is memory constrained compared to CPU rendering, and as soon as you try and go out-of-core on the GPU, you're back at CPU speeds, so there's usually no point doing GPU rendering for entire scenes (which can take > 48 GB of RAM for all geometry, accel structures, textures, etc) given the often large memory re…

> Because the expense is not really worth it I disagree with this takeaway. But full disclosure I’m biased: I work on OptiX. There is a reason Pixar and Arnold and Vray and most other major industry renderers are moving to the GPU, because the trends are clear and because it has recently become ‘worth it’. Many renderers are reporting factors of 2-10 for production scale scene rendering. (Here’s a good example: https…

I should also point out that ray traversal / intersection costs are generally only around 40% of the costs of extremely large scenes, and that's predominantly where GPUs are currently much faster than CPUs.

(I'm aware of the OSL batching/GPU work that's taking place, but it remains to be seen how well that's going to work).

From what I've heard from friends in the industry (at other companies) who are using GPU versions of Arnold, the numbers are no-where near as good as the upper numbers you're claiming when rendering at final fidelity (i.e. with AOVs and Deep output), so again, the use-cases - at least for high-end VFX with GPU - are still mostly for lookdev and lighting blocking iterative workflow from what I understand. Which is still an advantage and provides clear benefits in terms of iteration time over CPU renderers, but it's not a complete win, and so far, only the smaller studios have started dipping their toes in the water.

Also, the advent of AMD Epyc has finally thrown some competitiveness back to CPU rendering, so it's now possible to get a machine with x2 as many cores for close to half the price, which has given CPU rendering a further shot in the arm.

Re: Pixar's Render Farm

#134

Post by yiyus »

nikolay wrote: post #97
iambateman wrote: post #94

Earlier quoted context omitted.

We are way way past that point. It’s time for Twitter to evolve in so many ways.

I think they can't as their system is built with many of these limitations and preconceptions and it's hard for it to evolve easily.

It may be difficult to evolve, but it should be possible.

Re: Pixar's Render Farm

#135

Post by foota »

berkut wrote: post #125
dahart wrote: post #117

Earlier quoted context omitted.

> Because the expense is not really worth it I disagree with this takeaway. But full disclosure I’m biased: I work on OptiX. There is a reason Pixar and Arnold and Vray and most other major industry renderers are moving to the GPU, because the trends are clear and because it has recently become ‘worth it’. Many renderers are reporting factors of 2-10 for production scale scene rendering. (Here’s a good example: https…

I write a production renderer for a living :) So I'm well aware of the trade offs. As I mentioned, for lookdev and small scenes, GPUs do make sense currently (if you're willing the pay the penalty of getting code to work on both CPU and GPU, and GPU dev is not exactly trivial in terms of debugging / building compared to CPU dev). But until GPUs exist with > 64 GB RAM, for rendering large scale scenes, it's just not w…

Given current consumer GPUs are at 24 GB I think 3-4 years is likely overly pessimistic.

Re: Pixar's Render Farm

#137

Post by dodobirdlord »

brundolf wrote: post #29
chrisseaton wrote: post #24

Earlier quoted context omitted.

Why can’t it be simple wall-clock time? Each frame takes 24 hours of real wall-clock time to render start to finish. But they render multiple frames at the same time. Doing so does not change the wall-clock time of each frame.

In my (hobbyist) experience, path-tracing and rendering in general are enormously parallelizable. So if you can render X frames in parallel such that they all finish in 24 hours, that's roughly equivalent to saying you can render one of those frames in 24h/X. Of course I'm sure things like I/O and art-team-workflow hugely complicate the story at this scale, but I still doubt there's a meaningful concept of "wall-cloc…

Ray tracing is embarrassingly parallel, but it requires having most if not all of the scene in memory. If you have X,000 machines and X,000 frames to render in a day, it almost certainly makes sense to pin each render to a single machine to avoid having to do a ton of moving data around the network and in and out of memory on a bunch of machines. In which case the actual wall-clock time to render a frame on a single machine that is devoted to the render becomes the number to care about and to talk about.

Re: Pixar's Render Farm

#138

Post by tinco »

aprdm wrote: post #75
nom wrote: post #34

Oh man, I wanted this to contain much more details :( Whats the hardware? How much electric energy goes into rendering a frame or a whole movie? How do they provision it (as they keep #cores fixed)? They only talk about cores, do they even use GPUs? What's running on the machines? What did they optimize lately? So many questions! Maybe someone from Pixar's systems department is reading this :)?

Not Pixar specifically but Modern VFX and Animation studios usually have a bare metal render farm, they usually are pretty beefy -- think at least 24 cores / 128 GB of RAM per node. Usually in crunch time if there's not enough nodes in the render farm they might rent nodes connecting them to their network for a period of time, or they might use the cloud, or they might get budget to increase their render farms. From…

Can confirm cloud GPU is way overpriced if you're doing 24/7 rendering. We run a bare metal cluster (not VFX but photogrammetry) and I pitched our board on the possibilities. I really did not want to run a bare metal cluster, but it just does not make sense for a low margin startup to use cloud processing.

Running 24/7 for three months, it's cheaper to buy consumer grade hardware with similar (probably better) performance. "Industrial" grade hardware (Xeon/Epyc + Quadro) it's under 12 months. We chose consumer grade bare metal.

On thing that was half surprising, half calculated in our decision was despite the operational overhead how much less stressful running your own hardware is. When we ran experimentally on the cloud, a misrender could cost us 900 euro, and sometimes we'd have to render 3 times or more for a single client. Bringing us from healthily profitable to losing money. The stress of having to get it right the first time sucked.

Re: Pixar's Render Farm

#139

Post by berkut »

foota wrote: post #135
berkut wrote: post #125

Earlier quoted context omitted.

I write a production renderer for a living :) So I'm well aware of the trade offs. As I mentioned, for lookdev and small scenes, GPUs do make sense currently (if you're willing the pay the penalty of getting code to work on both CPU and GPU, and GPU dev is not exactly trivial in terms of debugging / building compared to CPU dev). But until GPUs exist with > 64 GB RAM, for rendering large scale scenes, it's just not w…

Given current consumer GPUs are at 24 GB I think 3-4 years is likely overly pessimistic.

They've been at 24 GB for two years though - and they cost an arm and a leg compared to a CPU with a similar amount.

It's not just about them existing, they need to be cost effective.

Post reply on HN