Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

Post reply on HN
Print view View original

151–160 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#151

Post by hinkley »

hellotomyrars wrote: post #8

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

Blizzard sorted the high res textures and some of the raids last so that you could start playing the game halfway through the download.

Another problem is chunking the data properly so that incremental updates are not hugs.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#152

Post by usefulcat »

kevingadd wrote: post #94
hellotomyrars wrote: post #8

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

Titanfall shipped uncompressed audio on purpose to cut out audio decompression CPU usage. You can disagree with the decision, but it wasn't an accident or negligence.

Are spare cores really such a scarce resource? I thought most games didn't come close to using all available cores. And audio decompression is not known for being that cpu intensive.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#153

Post by rvba »

hinkley wrote: post #151
hellotomyrars wrote: post #8

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

Blizzard sorted the high res textures and some of the raids last so that you could start playing the game halfway through the download. Another problem is chunking the data properly so that incremental updates are not hugs.

Blizzard us a bad example. Their games take incredible amounts of time to start. I think they dont play their own games and nobody gives a shit.

The launcher is bugged and downloads either empty or 137mb patches (maybe bad CRC at some server? Different CRCs at some?). Every time.

Starcraft remastered was bugged at some point and couldnt be played for a week.

If you statt Heroes of the storm, it takes few minutes to download some 600 byte ghost patch, then does some (antihack?) processing... then you can start the queue after like 10 minutes.

It's probably like the GTA5 that also took ages to start due to poorly done code and devs not caring at all.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#155

Post by dendriti »

bhouston wrote: post #154

250gB? No problem with 1gb/S to 3gb/s fiber to the home. At least here in Ottawa, Canada Fiber to the home is prevalent and affordable. But I understand where fiber isn’t available these are large downloads.

Well shit. Does your "Canada Fiber to the home" come with unlimited fast storage for free?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#156

Post by arthur2e5 »

topspin wrote: post #65
arthur2e5 wrote: post #53

Earlier quoted context omitted.

War Thunder kind of has that, except that the "progressive" bit means you need to download the full thing to progress in the game beyond a certain point. Enlisted, which uses the same engine, is fine though.

The design of ArenaNet's GW/GW2 has always impressed me. 99.4% of all of the game is contained in one file: GW2.DAT, which is (currently) 66.7 GB. I/O for updates is extremely efficient. I've seen many games create a vast file system mess, and they're forever plagued by "corruption" or something, complete with "fix" tools, "cache" clearing push-ups and players reinstalling after big updates that don't install correct…

> I/O for updates is extremely efficient.

This happens to touch upon a pain I keep getting with Steam. Games like squad keep getting tiny patches (compared to the whole game size), which is great, until you see that steam needs to read and write the whole file again to finish the patch. It's not as simple as their "dumb allocation wearing out SSD" problem either -- there's no API to do this anywhere unlike the common posix_fallocate(). Linux FALLOC_FL_INSERT_RANGE comes close, but no studio will tend their files so they make exact filesystem block-sized units.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#157

Post by langsoul-com »

pornel wrote: post #71

Compression optimizes for wall clock time which is (roughly speaking) sum of time it takes to get the compressed data and time it takes to decompress it. Downloads over the network are almost always bottlenecked on the network speed, so the heavier compression you have the better. However, when loading data from the local disk, the I/O can be so fast that the decompression time becomes the bottleneck and slows down l…

There's a limit to compression. At some point, too much compression takes too long to unpack. The normal person wouldn't really understand that.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#159

Post by sgtnoodle »

jjoonathan wrote: post #21
terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

In 2003, I found out that America's Army (the US Army videogame) had compressed and decompressed all their textures at some point, so they were paying the quality hit for compression and the space hit for RGBA. This was a time when the game still took a day to download, even on a half decent connection, so >200% footprint inflation kinda mattered. The more things change, the more they stay the same!

I have fond memories of that game. 80% of the time I would play very patiently and carefully. Occasionally, though, I would go Rambo and take out the whole enemy team in 30 seconds.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#160

Post by mirsadm »

kevingadd wrote: post #122
Sohcahtoa82 wrote: post #106

Earlier quoted context omitted.

What a bunch of absolute horse shit. MP3 isn't slow. I was playing MP3s on a 100 Mhz 486 DX4 in 1996. Certainly a system in 2014 can handle decompressing MP3s on the fly. Even if you didn't want to do it on the fly during game play, they easily could have been decompressed at load time and it would have taken only a couple seconds. It's not like ENTIRE 35 GB of audio needed to be loaded all at once, nobody had that R…

I've benchmarked it myself in my game. It adds a measurable amount of time to loads if you're decompressing all the audio up front to reduce latency, and during streaming it increases CPU usage. Obviously a modern machine can handle a few ogg/mp3 streams in the background during gameplay, but some games are playing dozens or hundreds of sounds at once. "It's not like ENTIRE 35 GB of audio needed to be loaded all at o…

Plus last gen consoles have extremely slow CPUs. They were already slow when they were released.
Post reply on HN