Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

Post reply on HN
Print view View original

91–100 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#91

Post by kevingadd »

xaduha wrote: post #50
taeric wrote: post #22

Earlier quoted context omitted.

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

> This feels wrong. In that, I'd be surprised if they are not using compression. Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ... from https://github.com/IridiumIO/CompactGUI

This list appears to be largely indie games, many with sizes under a gigabyte. I'm not sure why at-rest compression would be a high priority for a small team when their game fits on a $5 thumbdrive and will be compressed during the install process by Steam or Itch.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#92

Post by kevingadd »

terafo wrote: post #72
forrestthewoods wrote: post #57

Earlier quoted context omitted.

> It's not just speculation. So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades. Why are new games so large? Good question. This entire thread is speculative.

Jedi Survivor having uncompressed textures is a speculation, but I'm extremely confident that it is this way. The fact that games nowadays ship without LOD systems, without texture compression, without adequate streaming is not just speculation. It is a fact.

You're saying Jedi Survivor doesn't use block texture compression? On which target? All targets?

Can you capture a trace in RenderDoc to prove this? Or show raw bmp/uncompressed dds files on disk?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#93

Post by kevingadd »

gabereiser wrote: post #77
terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

This is mostly true. The packages of assets for games are compressed but the individual assets are usually not. It’s mostly fancy zip of files with maybe a different header.

So they're compressed? What's wrong with compressing packages? That usually produces a better compression ratio by exploiting redundancy cross-file.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#94

Post by kevingadd »

hellotomyrars wrote: post #8

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

Titanfall shipped uncompressed audio on purpose to cut out audio decompression CPU usage. You can disagree with the decision, but it wasn't an accident or negligence.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#95

Post by gatane »

GuB-42 wrote: post #45
LinuxBender wrote: post #11

I recall a lot of hype more than a decade ago about all the textures going away and being replaced with math formulas to generate everything on the fly. Games were supposed to get dramatically smaller. What ever happened to that?

And there was also the opposite, John Carmack in particular didn't have a problem with using huge amount of data, "Rage" is an application of this idea. The game uses a single, huge, non-repeating "megatexture", with elaborate caching strategies to keep it playable even on slow media (HDD/optical). The idea was to give complete freedom to artists and not require them to be mathematicians. Further down the line, he ev…

I see that Inigo Quilez work is more in the "render a scene not necesarily in real time" realm. Like, you could make a raytracing program, make it render off camera a shiny scene, and only after rendering it shows you the image.

Because of this, its very suited to math and procedural generation. You just change some parameters, render and show. Games instead have to do this in real time, with less latency and CPU/GPU usage. And procedural/math generated content tends to be CPU heavy (more usage+latency), if not also on the GPU (cant be cached, etc).

Shaders are more on the middle of this, because you want to apply certains transformations to and from rendering data.

Just my armchair observation.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#96

Post by kevingadd »

pornel wrote: post #71

Compression optimizes for wall clock time which is (roughly speaking) sum of time it takes to get the compressed data and time it takes to decompress it. Downloads over the network are almost always bottlenecked on the network speed, so the heavier compression you have the better. However, when loading data from the local disk, the I/O can be so fast that the decompression time becomes the bottleneck and slows down l…

Heavy compression for the first install is a common tactic. Steam and itch do it, and brotli transfer compression is widely used on the web.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#97

Post by tmilard »

egypturnash wrote: post #89
tmilard wrote: post #60

Chocked at FAT size video games file ? Me too. Giga bytes Damned ! To reduce the size, I decided : 1) To use the standard PNG compression. 2) To limit mesh size at less than 300 points for a scene. - Result : a) File size Check it here : https://free-visit.net/fr/demo01 (pleaseload url twice : I still have a bug) Question : - why video games STUDIOS do not use this strategy ?

To limit mesh size at less than 300 points for a scene. I am pretty sure most modern video games use more mesh points than that for, like, a hat your character can wear.

I it is true : I was very agressive on this one. But still, thousands and tousands of triangle feels like so much overweight for me. Specialy for indoor scenes. Quality for the eye is a lot to do with texture, NOT the number of triangle.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#98

Post by kevingadd »

topspin wrote: post #65
arthur2e5 wrote: post #53

Earlier quoted context omitted.

War Thunder kind of has that, except that the "progressive" bit means you need to download the full thing to progress in the game beyond a certain point. Enlisted, which uses the same engine, is fine though.

The design of ArenaNet's GW/GW2 has always impressed me. 99.4% of all of the game is contained in one file: GW2.DAT, which is (currently) 66.7 GB. I/O for updates is extremely efficient. I've seen many games create a vast file system mess, and they're forever plagued by "corruption" or something, complete with "fix" tools, "cache" clearing push-ups and players reinstalling after big updates that don't install correct…

A massive amount of engineering went into making it work and some of the stuff we did was downright evil :) in gw1, a lot of data was compiled into the exe as c constants so that we wouldn't have to do a bunch of file management and I/o to locate and load it. We ended up hitting limitations in the C compiler.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#99

Post by kevingadd »

tmilard wrote: post #60

Chocked at FAT size video games file ? Me too. Giga bytes Damned ! To reduce the size, I decided : 1) To use the standard PNG compression. 2) To limit mesh size at less than 300 points for a scene. - Result : a) File size Check it here : https://free-visit.net/fr/demo01 (pleaseload url twice : I still have a bug) Question : - why video games STUDIOS do not use this strategy ?

PNG decompression is way too slow. Some games use it but it's on the way out, either being replaced by codecs like Basis or QOI in most cases

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#100

Post by kevingadd »

tmilard wrote: post #69
codedokode wrote: post #38

Why not simply make several versions? Like 1 Gb, 10 Gb and 100 Gb versions?

good idea : one why chitty etxture quality, one with average, one with quality texture. They have done this on web-videos. Why not on video games files ?

They do, the Xbox calls it smart delivery.
Post reply on HN