How is there so much duplication?
The post stated that it was believed duplication improved loading times on computers with HDDs rather than SSDs
Key word is "believed". It doesn't sound like they actually benchmarked.
Helldivers 2 on-disk size 85% reduction
21–30 of 317 posts
Re: Helldivers 2 on-disk size 85% reduction
#22Re: Helldivers 2 on-disk size 85% reduction
#23Earlier quoted context omitted.
They duplicate files to reduce load times. Here's how Arrowhead Game Studios themselves tell it: https://www.arrowheadgamestudios.com/2025/10/helldivers-2-te...
I don't think this is the real explanation. If they gave the filesystem a list of files to fetch in parallel (async file IO), the concept of "seek time" would become almost meaningless. This optimization will make fetching from both HDDs and SSDs faster. They would be going out of their way to make their product worse for no reason.
Solid state drives tend to respond well to parallel reads, so it's not so clear. If you're reading one at a time, sequential access is going to be better though.
But for a mechanical drive, you'll get much better throughput on sequential reads than random reads, even with command queuing. I think earlier discussion showed it wasn't very effective in this case and taking 6x the space for a marginal benefit for the small % of users with mechanical drives isn't worth while...
Re: Helldivers 2 on-disk size 85% reduction
#24Earlier quoted context omitted.
Is it really possible to control file locations on HDD via Windows NTFS API?
No, not at all. But by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially without additional seeks. That does force you to duplicate some assets a lot. It's also more important the slower your seeks are. This technique is perfect for disc media, since it has a fixed physical size (so wasting space on it is irrelevant) and slow seeks.
> by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially
I'd love to see it analysed. Specifically, the average number of nonseq jumps vs overall size of the level. I'm sure you could avoid jumps within megabytes. But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower. This paper effectively says that if you have long files, there's almost guaranteed gaps https://dfrws.org/wp-content/uploads/2021/01/2021_APAC_paper... so at that point, you may be better off preallocating the individual does where eating the cost of switching between them.
Re: Helldivers 2 on-disk size 85% reduction
#25https://news.ycombinator.com/item?id=46134178
282 comments
Re: Helldivers 2 on-disk size 85% reduction
#26Earlier quoted context omitted.
No, not at all. But by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially without additional seeks. That does force you to duplicate some assets a lot. It's also more important the slower your seeks are. This technique is perfect for disc media, since it has a fixed physical size (so wasting space on it is irrelevant) and slow seeks.
> by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially I'd love to see it analysed. Specifically, the average number of nonseq jumps vs overall size of the level. I'm sure you could avoid jumps within megabytes. But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower. This paper…
Sure. I’ve seen people that do packaging for games measure various techniques for hard disks typical of the time, maybe a decade ago. It was definitely worth it then to duplicate some assets to avoid seeks.
Nowadays? No. Even those with hard disks will have lots more RAM and thus disk cache. And you are even guaranteed SSDs on consoles. I think in general no one tries this technique anymore.
Re: Helldivers 2 on-disk size 85% reduction
#27Earlier quoted context omitted.
They duplicate files to reduce load times. Here's how Arrowhead Game Studios themselves tell it: https://www.arrowheadgamestudios.com/2025/10/helldivers-2-te...
I don't think this is the real explanation. If they gave the filesystem a list of files to fetch in parallel (async file IO), the concept of "seek time" would become almost meaningless. This optimization will make fetching from both HDDs and SSDs faster. They would be going out of their way to make their product worse for no reason.
"97% of the time: premature optimization is the root of all evil."
Re: Helldivers 2 on-disk size 85% reduction
#28Earlier quoted context omitted.
Which is true. It’s an old technique going back to CD games consoles, to avoid seeks.
Is it really possible to control file locations on HDD via Windows NTFS API?
Not really. But when you write a large file at once (like with an installer), you'll tend to get a good amount of sequential allocation (unless your free space is highly fragmented). If you load that large file sequentially, you benefit from drive read ahead and OS read ahead --- when the file is fragmented, the OS will issue speculative reads for the next fragment automatically and hide some of the latency.
If you break it up into smaller files, those are likely to be allocated all over the disk; plus you'll have delays on reading because windows defender makes opening files slow. If you have a single large file that contains all resources, even if that file is mostly sequential, there will be sections that you don't need, and read ahead cache may work against you, as it will tend to read things you don't need.
Re: Helldivers 2 on-disk size 85% reduction
#29My takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild
Re: Helldivers 2 on-disk size 85% reduction
#30I wonder if a certain Amelie-clad repacker noticed the same reduction in their release of the same game.
Helldivers 2 is an always-online game, you won't find it cracked.