Faster LZ is not the answer to 150-250 GB video game downloads

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't use any compression beyond something like BC7. And some games are not only without any compression, they ACTIVELY duplicate their assets on disk so they load a bit faster on HDD. And these builds end up on modern consoles that have ludicrously fast SSDs.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#6

Post by aappleby » Fri, Apr 21, 2023, 8:40 PM UTC

adgjlsfhk1 wrote: post #3

I really want to see someone make a renderer that can work on compressed textures. No need to decompress if you can just read the data on the fly.

Renderers have been using compressed textures for literal decades. Not JPG, alas, because the compression formats have to support random access to individual texels.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#7

Post by rektide » Fri, Apr 21, 2023, 8:48 PM UTC

Richg42 is great & his pieces are fun to read.

I do think in this case he misses a bit of the point, goes too far:

> They are designed for fast loading and streaming off SSD's. Who cares about saving off a few hundred ms (or a second) when it takes hours or days to download the product onto the SSD?

Most games will have only one two or three downloads over the user's lifetime, and it'll never be instant, will always require some wait.

But we load textures many times if we keep playing a game.

Rich's overall point remains, but I think some balance of optimizing load speed is also very appreciated & very necessary.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#8

Post by hellotomyrars » Fri, Apr 21, 2023, 8:48 PM UTC

Game size is a perennial talking point and some games earn it and some games don’t.

There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall)

Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think this is the smart way to do it.

Examples of games shrinking I are hard to come by but special credit to IO interactive and Hitman 3.

The install size with all the content of the previous games included (If you own them) is less than the install size of the first game. That is impressive and commendable.

Games are weird and some people will look at the file size and think it means the game is better, just the way some people think games with more hours are better. The psychology of people who play video games is weird.

Duplication of assets on disk was a very weird byproduct of HDD file access conventions, but we shouldn’t need that anymore. There is still a shrinking, but valid crossover window between devs being able to assume all storage is solid state. I think things will get better but the debate of file size vs average/expects storage capacity vs Network speed will never end but there is certainly work to be done.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#9

Post by terafo » Fri, Apr 21, 2023, 8:49 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

And, besides, his solutions aren't really good. Radical solution would be shipping image generating model with the game and generating textures on the fly. That's how all texture-related data goes from being measured in hundreds of gigs to being measured in hundreds of megs. So basically becomes non-issue.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#10

Post by KMnO4 » Fri, Apr 21, 2023, 8:50 PM UTC

Games want to be more photorealistic and the hardware to fake it is getting faster and cheaper.

I remember in 2008 when Intel demoed ray tracing[0] for Quake Wars. It ran on a quad-socket (4 CPUs x 2.7GHz) at around 25 FPS, and even Intel admitted that ray tracing is just "brute forcing" the complex optical physics of a scene.

Considering between then and now, we have much prettier looking games that do NOT require ray tracing for their visual effects (and can run on much fewer CPU cycles), thanks to much more "clever" techniques and algorithms (DoF, bloom, subsurface scattering, PBR, etc), and if it weren't for the RTX graphics cards setting the precedent, there would probably be more research into making things prettier without having to "brute force" things.

Nowadays, you can drop a 100gb mesh into UE5 Nanite[1] and skip a lot of the work involved in making things pretty. You don't need to generate or even understand displacement maps if you can just add more polygons to the model. You don't need to billboard the mountain way in the distance if Unreal does it on the fly. Who cares if that means including a 250mb mesh in your assets, versus a 5mb HDRI.

So yes, producers have either become lazy or are simply unable to meet the demands of the hardware by coming up with better algorithms. That's unlikely to change until either hardware stops advancing and either A) can't run the latest games or B) games become too large that they're impossible to download.

[0]: https://www.gamedeveloper.com/programming/sponsored-feature-...

[1]: https://docs.unrealengine.com/5.0/en-US/nanite-virtualized-g...

[2]: https://twitter.com/mariobrothblog/status/163604089376436224...