Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

41–50 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#41

I recall a lot of hype more than a decade ago about all the textures going away and being replaced with math formulas to generate everything on the fly. Games were supposed to get dramatically smaller. What ever happened to that?

It is an order of magnitude more expensive to produce art assets that way than to just hire people familiar with photoshop and maya. Those procedurally generated assets in tech demos like .kkrieger were just that: tech demos.

Asset production is already the bulk of the cost of AAA video game production.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#42
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

I remember Fallout New Vegas was almost 50 GB because it was a parallel install of all language/localization files. Once you manually deleted the languages you didn't want it went back to like 10 GB.

And there was a hilarious situation with audio in Titanfall, which included 35 gigs of uncompressed audio(whole game took 48 gigs).

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#43
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

[deleted]

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#44
post #22

Earlier quoted context omitted.

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc... People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

That is a little surprising to me. Especially if we are talking about games that are delivered online. Thinking on it, I'm actually only surprised for online delivered games. I don't see any incentive to reduce size on delivered disks.

I'd expect that they use a compression that is good for transport, at transport time. I'm not too shocked that they have the assets in a way that is easy to use at runtime.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#45

I recall a lot of hype more than a decade ago about all the textures going away and being replaced with math formulas to generate everything on the fly. Games were supposed to get dramatically smaller. What ever happened to that?

And there was also the opposite, John Carmack in particular didn't have a problem with using huge amount of data, "Rage" is an application of this idea. The game uses a single, huge, non-repeating "megatexture", with elaborate caching strategies to keep it playable even on slow media (HDD/optical). The idea was to give complete freedom to artists and not require them to be mathematicians. Further down the line, he even considered using sparse voxel octrees, i.e. even more huge 3D textures.

On the opposite end of the spectrum, you have Inigo Quilez ( https://iquilezles.org/ ). He literally draws with maths, and he is famous for his 4k demoscene productions. The problem is that about no one else is capable of doing what he does.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#46
post #12

The Moore's Law is Dead podcast recently did a podcast with an Infinity Ward award-winning effects artist: https://www.youtube.com/watch?v=tmfHxJT1I3I . The crazy thing he mentioned is that if there was zero compression, the latest COD games genuinely had over 1 terabyte of uncompressed assets.

But why?

Listen to the answer to the question, "How long until our game installs take up 500 gigabytes of space": https://youtu.be/tmfHxJT1I3I?t=7152s

The reality is a game has increasingly more content nowadays. Remember, there's a lot of "games as a service". So you have many "seasons" of additional content. You may not play all the content, but someone is. And when people want 4K and 8K textures, different variations so effects don't look the same twice, on top of all the other sizes, plus all the other effects ... it adds up. Never mind huge open worlds, where the asset footprint for just the map might be 100GB.

Having been in a few AAA studios myself (doing database stuff) and sitting next to artists and even art directors, the sheer amount of effort, headcount, etc. put into the art pipeline is genuine madness. In a large studio, art is over HALF the production cost, sometimes more towards 60-65%. Heck, one of the things sometimes brought up about Red Dead Redemption II is that there were three full-time employees in which all they were responsible for was horses. 6000 man-hours a year, on just horses.

To sum up what Chris said towards the end, "The only thing keeping these numbers down is the compression algorithms. If we didn't have these newer compression algorithms, we'd probably legitimately be at a terabyte. I can actually tell you for a fact, that there are over a terabyte of assets in the COD engine."

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#47

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

The .kkreiger devs really showed how far you can go to reduce download size:

https://en.wikipedia.org/wiki/.kkrieger

https://www.youtube.com/watch?v=ya_MUKc343U

It is dated, but looks pretty good by 2004 standards.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#48
post #9
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

And, besides, his solutions aren't really good. Radical solution would be shipping image generating model with the game and generating textures on the fly. That's how all texture-related data goes from being measured in hundreds of gigs to being measured in hundreds of megs. So basically becomes non-issue.

That would be far, far slower than just downloading the image

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#49
post #33

Earlier quoted context omitted.

But why?

Have you played a modern game? they have amazing levels of detail in the tiniest places. One game I play, Gran Turismo 7, claims that a single car model start to finish takes months to model for a single modeler. Some of the details in those cars that you can see in VR you can absolutely not see in-game normally, such as the door lock controls or the key-hole for the driver's door being exactly like on the real car,…

Gran Turismo in VR is an amazing experience. I'm not even a racing car fan, but holy crap does that game really capture an experience. I'm somewhat afraid to find out just how much data goes into a single model.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#50
post #22
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

> This feels wrong. In that, I'd be surprised if they are not using compression.

Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ...

from https://github.com/IridiumIO/CompactGUI

Post reply on HN