Faster LZ is not the answer to 150-250 GB video game downloads • Hacker News Board

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#21

Post by jjoonathan » Fri, Apr 21, 2023, 8:58 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

In 2003, I found out that America's Army (the US Army videogame) had compressed and decompressed all their textures at some point, so they were paying the quality hit for compression and the space hit for RGBA. This was a time when the game still took a day to download, even on a half decent connection, so >200% footprint inflation kinda mattered. The more things change, the more they stay the same!

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#22

Post by taeric » Fri, Apr 21, 2023, 8:58 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are.

Heck, just the geometry is probably far larger than folks anticipate.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#23

Post by bentcorner » Fri, Apr 21, 2023, 9:01 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

Agree - OP even links to this[1] PC gamer article which calls out Hitman 3 as having a much smaller size compared to Hitman 2 (and Hitman 3 includes all of Hitman 2). How did they do it? Compression[2].

[1] https://www.pcgamer.com/biggest-game-install-sizes/

[2] https://www.pcgamer.com/how-hitman-3s-devs-shrank-the-entire...

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#24

Post by yjftsjthsd-h » Fri, Apr 21, 2023, 9:01 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

> they ACTIVELY duplicate their assets on disk so they load a bit faster on HDD

How does that work?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#25

Post by sudosysgen » Fri, Apr 21, 2023, 9:03 PM UTC

yjftsjthsd-h wrote: post #24

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

> they ACTIVELY duplicate their assets on disk so they load a bit faster on HDD How does that work?

Shorter seek times because the bits are likely to be on the same or nearby tracks.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#26

Post by terafo » Fri, Apr 21, 2023, 9:04 PM UTC

KMnO4 wrote: post #10

Games want to be more photorealistic and the hardware to fake it is getting faster and cheaper. I remember in 2008 when Intel demoed ray tracing[0] for Quake Wars. It ran on a quad-socket (4 CPUs x 2.7GHz) at around 25 FPS, and even Intel admitted that ray tracing is just "brute forcing" the complex optical physics of a scene. Considering between then and now, we have much prettier looking games that do NOT require r…

> and if it weren't for the RTX graphics cards setting the precedent, there would probably be more research into making things prettier without having to "brute force" things.

Not really. A lot of this research included making rudimentary versions of RTGI that could run on hardware of that age(see SVOGI). Of course you can bake everything and make amazing looking levels, that are completely static. But that doesn't work with open-world games, especially if they have different times of day. And it doesn't work in highly dynamic scenes. We basically hit the end of the road for scaling graphics without RT. And now we have hardware that can do realtime path-traced rendering of major AAA titles, and it looks miles better than anything smoke and mirrors could've ever delivered.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#27

Post by sudosysgen » Fri, Apr 21, 2023, 9:04 PM UTC

taeric wrote: post #22

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc...

People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#28

Post by babypuncher » Fri, Apr 21, 2023, 9:05 PM UTC

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

There's no excuse for Jedi Survivor not to compress it's assets. Respawn is skipping last-gen consoles with this game. The modern consoles have significantly more CPU power at their disposal, and dedicated hardware for decompressing texture and sound data.

There is no excuse for them to be shipping a 150GB game on the PS5 when so many cross-gen games are 40-60% smaller on PS5 than PS4

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#29

Post by marcellus23 » Fri, Apr 21, 2023, 9:05 PM UTC

taeric wrote: post #22

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#30

Post by babypuncher » Fri, Apr 21, 2023, 9:06 PM UTC

overthrow wrote: post #20

terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

Jedi Survivor isn't shipping on any platforms constrained by an HDD, so it would be odd if they were making any optimizations for spinning rust, especially when they come at the cost of precious SSD space.