Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

Post reply on HN
Print view View original

51–60 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#51

Post by coffeebeqn »

taeric wrote: post #44
sudosysgen wrote: post #27

Earlier quoted context omitted.

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc... People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

That is a little surprising to me. Especially if we are talking about games that are delivered online. Thinking on it, I'm actually only surprised for online delivered games. I don't see any incentive to reduce size on delivered disks. I'd expect that they use a compression that is good for transport, at transport time. I'm not too shocked that they have the assets in a way that is easy to use at runtime.

Games are duct taped together at last minute on impossible deadlines pretty much every time.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#52

Post by coffeebeqn »

sudosysgen wrote: post #34
sroussey wrote: post #32

Earlier quoted context omitted.

Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.

Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.

Then you have to have the game reprocess all the assets on the users machine which could easily take an hour

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#53

Post by arthur2e5 »

topspin wrote: post #35
bick_nyers wrote: post #4

Faster internet and cheaper storage is the answer :) Or at the very least just offer two different downloads of the game, one with lossless textures and one with lossy textures...

Guild Wars 2 and others solve this with progressive downloads. You may start playing after a fraction of the content is downloaded. There is effectively no downside: new players can't access most of the content anyhow. It also applies to updates: you can start playing after a big update with a partial download. The background process can be safely interrupted and resumed without glitches.

War Thunder kind of has that, except that the "progressive" bit means you need to download the full thing to progress in the game beyond a certain point.

Enlisted, which uses the same engine, is fine though.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#54

Post by taeric »

xaduha wrote: post #50
taeric wrote: post #22

Earlier quoted context omitted.

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

> This feels wrong. In that, I'd be surprised if they are not using compression. Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ... from https://github.com/IridiumIO/CompactGUI

Thinking more on it, I am not too shocked that they would use uncompressed assets "at rest" on the machine. This is especially true for games mailed out on disks or cartridges. I'm still a little surprised, so the links are neat to read on.

I am kind of curious on why they wouldn't use compression in transport, as it were. I'm still mostly assuming that they do.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#55

Post by lpghatguy »

sudosysgen wrote: post #34
sroussey wrote: post #32

Earlier quoted context omitted.

Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.

Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.

You can combine the approaches! KTX2, for example, supports "supercompressing" GPU block compressed formats with LZ4 or Zstandard. The project I have been working on ships BCn textures compressed with zstd and it has been pretty dang good.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#56

Post by terafo »

sroussey wrote: post #32
sudosysgen wrote: post #27

Earlier quoted context omitted.

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc... People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.

You can add lossless compression, like Kraken, on top of that. PS5 has hardware accelerated Kraken decompression. And you can encode textures for GPU in a way that is better compressable with lossless algorithms.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#57

Post by forrestthewoods »

marcellus23 wrote: post #29
taeric wrote: post #22

Earlier quoted context omitted.

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.

> It's not just speculation.

So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades.

Why are new games so large? Good question. This entire thread is speculative.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#58

Post by blibble »

overthrow wrote: post #20
terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

if it's on a modern SSD you can DMA it directly into the GPU

no CPU required

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#60

Post by tmilard »

Chocked at FAT size video games file ? Me too. Giga bytes Damned !

To reduce the size, I decided : 1) To use the standard PNG compression. 2) To limit mesh size at less than 300 points for a scene.

- Result : a) File size Check it here : https://free-visit.net/fr/demo01 (pleaseload url twice : I still have a bug)

Question : - why video games STUDIOS do not use this strategy ?

Post reply on HN