Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helldivers 2 on-disk size 85% reduction

store.steampowered.com

Post reply on HN
Print view View original

31–40 of 317 posts

Re: Helldivers 2 on-disk size 85% reduction

#31

Post by ukd1 »

Xss3 wrote: post #15
imtringued wrote: post #11

Earlier quoted context omitted.

I don't think this is the real explanation. If they gave the filesystem a list of files to fetch in parallel (async file IO), the concept of "seek time" would become almost meaningless. This optimization will make fetching from both HDDs and SSDs faster. They would be going out of their way to make their product worse for no reason.

If they fill your harddrive youre less likely to install other games. If you see a huge install size youre less likely to uninstall with plans to reinstall later because thatd take a long time.

Unfortunately this actually is believable. SMH.

Re: Helldivers 2 on-disk size 85% reduction

#32

Post by maccard »

_aavaa_ wrote: post #29

My takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild

I've been involved in decisions like this that seem stupid and obvious. There's a million different things that could/should be fixed, and unless you're monitoring this proactively you're unlikely to know it hsould be changed.

I'm not an arrowhead employee, but my guess is at some point in the past, they benchmarked it, got a result, and went with it. And that's about all there is to it.

Re: Helldivers 2 on-disk size 85% reduction

#33

Post by toast0 »

viraptor wrote: post #24
dontlaugh wrote: post #19

Earlier quoted context omitted.

No, not at all. But by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially without additional seeks. That does force you to duplicate some assets a lot. It's also more important the slower your seeks are. This technique is perfect for disc media, since it has a fixed physical size (so wasting space on it is irrelevant) and slow seeks.

> by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially I'd love to see it analysed. Specifically, the average number of nonseq jumps vs overall size of the level. I'm sure you could avoid jumps within megabytes. But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower. This paper…

From that paper, table 4, large files had an average # of fragments around 100, but a median of 4 fragments. A handful of fragments for a 1 GB level file is probably a lot less seeking than reading 1 GB of data out of a 20 GB aggregated asset database.

But it also depends on how the assets are organized, you can probably group the level specific assets into a sequential section, and maybe shared assets could be somewhat grouped so related assets are sequential.

Re: Helldivers 2 on-disk size 85% reduction

#34

Post by maccard »

habbekrats wrote: post #2

it seems wild the state of games and development today... imagine 131GB out of 154GB of data was not needed....

This isn't unique to games, and it's not just "today". Go back a decade [0] find people making similar observations about one of the largest tech companies on the planet.

[0] https://news.ycombinator.com/item?id=10066338

Re: Helldivers 2 on-disk size 85% reduction

#35

Post by crest »

BizarroLand wrote: post #16
tetris11 wrote: post #12

Earlier quoted context omitted.

In terms of ownership, yes absolutely. In terms of read/write speeds to physical media, the switch to an SSD has been unsung gamechanger. That being said, cartridges were fast. The move away from cartridges was a wrong turn

I hate it when you buy a physical game, insert the disk, and immediately have to download the game in order to play the game because the disk only contains a launcher and a key. Insanity of the worst kind.

Or the launch day patch is >80% the size of the game, but I don't want to go back to game design limited by optical media access speeds.

Re: Helldivers 2 on-disk size 85% reduction

#36

Post by wcoenen »

viraptor wrote: post #24
dontlaugh wrote: post #19

Earlier quoted context omitted.

No, not at all. But by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially without additional seeks. That does force you to duplicate some assets a lot. It's also more important the slower your seeks are. This technique is perfect for disc media, since it has a fixed physical size (so wasting space on it is irrelevant) and slow seeks.

> by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially I'd love to see it analysed. Specifically, the average number of nonseq jumps vs overall size of the level. I'm sure you could avoid jumps within megabytes. But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower. This paper…

> But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower.

By default, Windows automatically defragments filesystems weekly if necessary. It can be configured in the "defragment and optimize drives" dialog.

Re: Helldivers 2 on-disk size 85% reduction

#37

Post by maccard »

Calzifer wrote: post #10

I was curious if they optimized the download. Did it download the 'optimized' ~150 GB and wasting a lot of time there or did it download the ~20 GB unique data and duplicated as part of the installation. I still don't know but found instead an interesting reddit post were users found and analyzed this "waste of space" three month ago. https://www.reddit.com/r/Helldivers/comments/1mw3qcx/why_the... PS: just found it.…

Steam breaks your content into 1MB Chunks and compresses/dedupes them [0]

[0] https://partner.steamgames.com/doc/sdk/uploading#AppStructur...

Re: Helldivers 2 on-disk size 85% reduction

#38

Post by crest »

andrewstuart wrote: post #4

How is there so much duplication?

The idea is to duplicate assets so loading a "level" is just sequential reading from the file system. It's required on optical media and can be very useful on spinning disks too. On SSDs it's insane. The logic should've been the other way around. Do a speed test on start an offer to "optimise for spinning media" if the performance metrics look like it would help.

If the game was ~20GB instead of ~150GB almost no player with the required CPU+GPU+RAM combination would be forced to put it on a HDD instead of a SSD.

Re: Helldivers 2 on-disk size 85% reduction

#39

Post by esrauch »

_aavaa_ wrote: post #29

My takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild

It's very easy to accidentally get misleading benchmarking results in 100 different ways, I wouldn't assume they did no benchmarking when they did the duplication.

Re: Helldivers 2 on-disk size 85% reduction

#40

Post by MLgulabio »

imtringued wrote: post #11
breve wrote: post #6

Earlier quoted context omitted.

They duplicate files to reduce load times. Here's how Arrowhead Game Studios themselves tell it: https://www.arrowheadgamestudios.com/2025/10/helldivers-2-te...

I don't think this is the real explanation. If they gave the filesystem a list of files to fetch in parallel (async file IO), the concept of "seek time" would become almost meaningless. This optimization will make fetching from both HDDs and SSDs faster. They would be going out of their way to make their product worse for no reason.

[dead]
Post reply on HN