Faster LZ is not the answer to 150-250 GB video game downloads

Post by toast0 » Sat, Apr 22, 2023, 11:39 PM UTC

blibble wrote: post #58

overthrow wrote: post #20

Earlier quoted context omitted.

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

if it's on a modern SSD you can DMA it directly into the GPU no CPU required

Post by rasz » Sun, Apr 23, 2023, 2:57 PM UTC

dontlaugh wrote: post #258

gabereiser wrote: post #131

Earlier quoted context omitted.

When optimizing load times, it’s usually console that’s the benchmark. PC is assumed to have faster or equal hardware so they optimize for the lowest denominator. Consoles aren’t PC’s but they are very close. They lack some instructions while gaining others. I can’t speak on current gen consoles but older ones like the Xbox 360, only could handle zip format and at a slow pace. You could show a loading screen while yo…

It does vary a bit. One exception is that current consoles guarantee fast SSDs and DMA to GPU, which can't be said of PCs.

Post by ehaliewicz2 » Mon, Apr 24, 2023, 8:55 PM UTC

ehaliewicz2 wrote: post #139

kevingadd wrote: post #82

Earlier quoted context omitted.

Saying "They take hundreds of gigs because they don't use ANY compression." is absurd when you immediately have to caveat that codecs like BC7 are used for texture data, which makes up a large portion of the shipped data. Sony included dedicated compression support in the PS5, licensed from a leading middleware company, specifically to make it easier to use compression in games. Games are using it. I don't know why y…

>(the most important thing, since raw assets often load faster from an SSD as long as you have available storage.) >You can argue every single file should be compressed at rest on persistent storage after install, but that's a losing argument. People really hate longer load times, so compression at rest is only appropriate if it improves them. Is this really true? I feel like you'd need an absolutely blisteringly fas…

Post by arthur2e5 » Thu, Apr 27, 2023, 7:47 AM UTC

arthur2e5 wrote: post #156

topspin wrote: post #65

Earlier quoted context omitted.

The design of ArenaNet's GW/GW2 has always impressed me. 99.4% of all of the game is contained in one file: GW2.DAT, which is (currently) 66.7 GB. I/O for updates is extremely efficient. I've seen many games create a vast file system mess, and they're forever plagued by "corruption" or something, complete with "fix" tools, "cache" clearing push-ups and players reinstalling after big updates that don't install correct…

> I/O for updates is extremely efficient. This happens to touch upon a pain I keep getting with Steam. Games like squad keep getting tiny patches (compared to the whole game size), which is great, until you see that steam needs to read and write the whole file again to finish the patch. It's not as simple as their "dumb allocation wearing out SSD" problem either -- there's no API to do this anywhere unlike the common…