IdTech 4, 15% frame rate increase through semiautomatic paralellization

IdTech 4, 15% frame rate increase through semiautomatic paralellization
vectorfabrics.com

From looking at the patch (nice of them to provide it), it seems like most of the changes are putting mutexes around things to guard against simultaneous access and then parallelizing some loops. It seems to me like anyone with basic knowledge of the game's codebase could have made these changes without any support from a tool like the one they're trying to sell, and probably done it in far less than 3 weeks. If your goal is to run a piece of code in parallel, you don't need a program to walk through that code and identify all the shared state it manipulates (and then go through and make sure that state has a mutex guard). If the software they're trying to sell did that for you, now that would be nice. It seems unlikely that you could automatically modify C++ source code without a lot of constraints, though.

Software to aid authoring parallel C++, or detect places where you can safely parallelize existing code, would certainly be valuable but this is a pretty poor demonstration of any actual advantages offered by their software.

EDIT: From watching the included demo video, the game is running well below 60fps which suggests that they're running it on a low end graphics card (or generally low-clocked machine) to improve the gains from their optimizations. I don't think their results are necessarily real-world as a result of this - someone playing Doom 3 on an actual gaming rig would probably not see a 15% framerate increase because their framerate would already be well above 60fps (framerate is nonlinear, as I mentioned above).

1. The allocator was a bottleneck; wonder if tcmalloc would have done better?

2. Computing what to draw? (idInteraction) was reduced to a for-loop and parallelized

3. ... lots more pre-rendering stuff ...

Unfortunately they didn't annotate the patch with the size of the win for any given change, or write up an analysis on what the changes were (I'm unfamiliar with idTech, too).

If they got the biggest win from improving the allocator, well, that's not very interesting. If they got the biggest win from the other stuff, and the tool told them what was safe to parallelize then that's much more interesting.

Actually, the whole thing would be better if the write up explained how they used the tool and what it did and how much improvement they got at each step...

Instead of spending 3 weeks wrapping data structures in mutexes for a 15% framerate increase, you could probably get an equivalent speedup by reducing shader detail, texture size, or any number of other options. This kind of measurement is not meaningful without additional information: Is this improved performance for low-spec GPUs? High-spec CPUs? universal performance improvement for all users? Low end machines typically do not have lots of cores; will they actually benefit tremendously from parallelism? As I said before, a high-spec machine already runs Doom 3 fine, so I question whether the gains here would actually be significant at all.

EDIT: Also, gains of this size can easily be produced by changes as simple as turning on Profile-Guided Optimization in your compiler. We'd also need to know what settings they used to compile the game to know whether the results are actually realistic.

When considering performance issues for a particular customer, you have to weigh the cost of improvement. 3 weeks of an engineer's time for 15% speedup (even if it were in elapsed frame time, which it's not) is NOT an easy decision. Those 3 weeks could be spent doing far more valuable things, like improving the tools used by the rest of the development team, fixing crash bugs, or working on downloadable content that will bring in more money for the studio. They could also be spent making architectural changes that provide much larger performance wins through design improvements that require actual knowledge about the design of the application and the significance of the choices made.

When you're doing performance optimization in game development, you go for the biggest, cheapest wins first, based on actual measurements and an understanding of what's wrong. Given that the optimized version of the game is not running at 60fps (and the demo video contains obvious rendering glitches) I don't think it's unfair to question whether CPU parallelism was the most obvious bottleneck here.

Lastly, dropping texture size or shader detail when running on a low spec machine is not a regression. If the machine is not capable of running at max rendering detail due to GPU constraints, no degree of CPU parallelism will overcome this.