Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

61–70 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#61

I really want to see someone make a renderer that can work on compressed textures. No need to decompress if you can just read the data on the fly.

https://www.reedbeta.com/blog/understanding-bcn-texture-comp...

BC1-5 have been standard practice since before the original XBox. I think when people here are talking about "uncompressed assets" they mean "not applying variable-rate zip style compression on top of BCn compression."

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#62
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

Agree - OP even links to this[1] PC gamer article which calls out Hitman 3 as having a much smaller size compared to Hitman 2 (and Hitman 3 includes all of Hitman 2 ). How did they do it? Compression[2]. [1] https://www.pcgamer.com/biggest-game-install-sizes/ [2] https://www.pcgamer.com/how-hitman-3s-devs-shrank-the-entire...

There was more to it than just compression. Hitman 2 allowed the player to install/uninstall each level individually, and the way that was implemented meant that each level had its own copy of every asset it contained. Hitman 3 removed that option and simply shipped every level in one unified package, with assets used in multiple levels only stored once.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#63

Earlier quoted context omitted.

Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.

Then you have to have the game reprocess all the assets on the users machine which could easily take an hour

Essentially any modern image compression format is going to be limited, or close to, by storage speed (for decompression). So unless the user has insanely fast internet (and the servers are also able to saturate their connection), it will be faster in the end.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#65
post #35

Earlier quoted context omitted.

Guild Wars 2 and others solve this with progressive downloads. You may start playing after a fraction of the content is downloaded. There is effectively no downside: new players can't access most of the content anyhow. It also applies to updates: you can start playing after a big update with a partial download. The background process can be safely interrupted and resumed without glitches.

War Thunder kind of has that, except that the "progressive" bit means you need to download the full thing to progress in the game beyond a certain point. Enlisted, which uses the same engine, is fine though.

The design of ArenaNet's GW/GW2 has always impressed me. 99.4% of all of the game is contained in one file: GW2.DAT, which is (currently) 66.7 GB. I/O for updates is extremely efficient.

I've seen many games create a vast file system mess, and they're forever plagued by "corruption" or something, complete with "fix" tools, "cache" clearing push-ups and players reinstalling after big updates that don't install correctly for whatever reason. Somehow ArenaNet surrounded this whole problem and solved it optimally.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#67
post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

In terms of attracting engineering talent, how does the video games industry fare?

In a crunch situation, I can think of how easy it is for some to be able to circumnavigate various pitfalls while others fall into them and have no time to address it.

Very much a skill/experience thing. There’s countless examples of games that run horribly given their level of presentation, or have no business being so impressively performant on the hardware they’re running.

I can guess what execs will pick when faced with “delay the game or ship a massive bundle.”

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#68
post #9

Earlier quoted context omitted.

And, besides, his solutions aren't really good. Radical solution would be shipping image generating model with the game and generating textures on the fly. That's how all texture-related data goes from being measured in hundreds of gigs to being measured in hundreds of megs. So basically becomes non-issue.

That would be far, far slower than just downloading the image

8k texture with mipmaps takes around 100 megs uncompressed. Downloading it would be 8-10 seconds on 100 mbit connection. It is definitely possible to generate that image in 8 to 10 seconds on consumer GPU if you are using some kind of consistency models that are trained to generate specific game style and inferences in 8 bit.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#69

Why not simply make several versions? Like 1 Gb, 10 Gb and 100 Gb versions?

good idea : one why chitty etxture quality, one with average, one with quality texture. They have done this on web-videos. Why not on video games files ?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#70

Earlier quoted context omitted.

Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.

> It's not just speculation. So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades. Why are new games so large? Good question. This entire thread is speculative.

Um, no. It's not speculation. Many games have been shown to use uncompressed assets, that's an objective fact.

The thread might be speculating about _why_ they do that, or whether this particular game does that, but that's a non sequitur, that isn't what I was talking about.

Post reply on HN