Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helldivers 2 devs slash install size from 154GB to 23GB

tomshardware.com

Post reply on HN
Print view View original

171–180 of 297 posts

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#171

Post by icecube123 »

dotwaffle wrote: post #89
fleabitdev wrote: post #51

Earlier quoted context omitted.

In this case, the bug was 131 GB of wasted disk space after installation. Because the waste came from duplicate files, it should have had little impact on download size (unless there's a separate bug in the installer...) This is why the cost of the bug was so easy for the studio to ignore. An extra 131 GB of bandwidth per download would have cost Steam several million dollars over the last two years, so they might ha…

> An extra 131 GB of bandwidth per download would have cost Steam several million dollars over the last two years Nah, not even close. Let's guess and say there were about 15 million copies sold. 15M * 131GB is about 2M TB (2000 PB / 2 EB). At 30% mean utilisation, a 100Gb/s port will do 10 PB in a month, and at most IXPs that costs $2000-$3000/month. That makes it about $400k in bandwidth charges (I imagine 90%+ is…

Yea, I always laugh when folks talk about how expensive they claim bandwidth is for companies. Large “internet” companies are just paying a small monthly cost for transit at an IX. They arent paying $xx/gig ($1/gig) like the average consumer is. If you buy a 100gig port for $2k, it costs the same if you’re using 5 GB a day or 8 PB per day.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#172

Post by cratermoon »

embedding-shape wrote: post #155
Capricorn2481 wrote: post #140

Earlier quoted context omitted.

Premature optimization doesn't mean "We have an obvious fix sitting in front of us that will definitely improve things." It means "We think we have something that could help performance based on a dubiously applicable idea, but we have no real workload to measure it on. But we're going to do it anyway." So it doesn't save us from anything, it potentially delays launching and gives us the same result that product team…

Yes, you and me understand that quote, probably mostly because we've both read all the text around the quote too, not just the quote itself. But there is a lot of people who dogmatically follow things other's write about without first digging deeper, and it's these people I was talking about before. Lots of people seemingly run on whatever soundbites they can remember.

While I know the paper pretty well, I still tend to phrase my objections by asking something along the lines of "do you have any benchmarks for the effects of that change?"

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#173

Post by superkuh »

PunchyHamster wrote: post #146
superkuh wrote: post #135

Earlier quoted context omitted.

It is no surprise to me that people still have to use HDD for storage. SSD stopped getting bigger a decade plus ago. SSD sizes are still only equal to the HDD sizes available and common in 2010 (a couple TB~). SSD size increases (availability+price decreases) for consumers form factors have entirely stopped. There is no more progress for SSD because quad level cells are as far as the charge trap tech can be pushed an…

I bought 4x (1TB->4TB the storage for half the price after my SSD died after 5 years (thanks samsung), what you mean they 'stopped being bigger'? Sure, there is some limitation in format, can only shove so many chips on M.2, but you can get U.2 ones that are bigger than biggest HDD (tho price is pretty eye-watering)

By stopped getting bigger I mean people still think 4TB is big in 2025. Just like 2010 when 3/4TB was the max size for consumer storage devices. u.2/u.3 is not consumer yet, unfortunately. I have to use m.2 nvme to u.2 adapters which are not great. And as you say, low number of consumer cpu+mobo pcie lanes has been restricting from the number of disks side until just recently. At least in 2025 we can have more than 2 nvme storage disks again without disabling a pcie slot.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#174

Post by loeg »

superkuh wrote: post #135
deng wrote: post #31

Earlier quoted context omitted.

11% still play HD2 with a spinning drive? I would've never guessed that. There's probably some vicious circle thing going on: because the install size is so big, people need to install it on their secondary, spinning drive...

It is no surprise to me that people still have to use HDD for storage. SSD stopped getting bigger a decade plus ago. SSD sizes are still only equal to the HDD sizes available and common in 2010 (a couple TB~). SSD size increases (availability+price decreases) for consumers form factors have entirely stopped. There is no more progress for SSD because quad level cells are as far as the charge trap tech can be pushed an…

SSDs did not "stop getting bigger a decade plus ago." The largest SSD announced in 2015 was 16TB. You can get 128-256TB SSDs today.

You can buy 16-32TB consumer SSDs on NewEgg today. Or 8TB in M.2 form factor. In 2015, the largest M.2 SSDs were like 1TB. That's merely a decade. At a decade "plus," SSDs were tiny as recently as 15 years ago.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#175

Post by JohnBooty »

ghurtado wrote: post #160
jeroenhd wrote: post #122

Earlier quoted context omitted.

They used "industry data" to make performance estimations: https://store.steampowered.com/news/app/553850/view/49158394... > These loading time projections were based on industry data - comparing the loading times between SSD and HDD users where data duplication was and was not used. In the worst cases, a 5x difference was reported between instances that used duplication and those that did not.

>In the worst cases, a 5x difference was reported between instances that used duplication and those that did not. Never trust a report that highlights the outliers before even discussing the mean. Never trust someone who thinks that is a sane way to use of statistics. At best they are not very sharp, and at worst they are manipulating you. > We were being very conservative and doubled that projection again to account…

Are you a working software engineer?

I'm sure that whatever project you're assigned to has a lot of optimization stuff in the backlog that you'd love to work on but haven't had a chance to visit because bugfixes, new features, etc. I'm sure the process at Arrowhead is not much different.

For sure, duplicating those assets on PC installs turned out to be the wrong call.

But install sizes were still pretty reasonable for the first 12+ months or so. I think it was ~40-60GB at launch. Not great but not a huge deal and they had mountains of other stuff to focus on.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#176

Post by zelphirkalt »

horsawlarway wrote: post #125
zelphirkalt wrote: post #116

Earlier quoted context omitted.

Which goes to show, that they don't care about the user, but only about the user's money.

No - because most users also don't check install size on games, and unlike renting overpriced storage from a cloud provider, users paid a fixed price for storage up front and aren't getting price gouged nearly as badly. So it's a trade that makes sense. Both entrants in the market are telling you that "install size isn't that important". If you asked the player base of this game whether they'd prefer a smaller size,…

So guide me through this thought process:

You are saying, that most users don't check install size of their games. Which I am not convinced of, but might even be true. Lets assume this to be true for the moment. How does this contradict, what I stated? How does users being uninformed or unaware of technical details make it so that suddenly cramming the user's disk is "caring" instead of "not caring"? To me this does not compute. Users will simply have a problem later, when their TBs of disk space have been filled with multiple such disk space wasters. Wasting this much space is user-hostile.

Next you are talking about _content_, which most likely doesn't factor in that much at all. Most of that stuff is high resolution textures, not content. It's not like people are getting significantly more content for bigger games. It is graphics craze, that many people don't even need. I am still running around with 2 full-HD screens, and I don't give a damn about 4k resolution textures. I suspect that a big number of users doesn't have the hardware to run modern games fluently at 4k.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#177

Post by icecube123 »

stanac wrote: post #104
dotwaffle wrote: post #89

Earlier quoted context omitted.

> An extra 131 GB of bandwidth per download would have cost Steam several million dollars over the last two years Nah, not even close. Let's guess and say there were about 15 million copies sold. 15M * 131GB is about 2M TB (2000 PB / 2 EB). At 30% mean utilisation, a 100Gb/s port will do 10 PB in a month, and at most IXPs that costs $2000-$3000/month. That makes it about $400k in bandwidth charges (I imagine 90%+ is…

Off topic question. > I imagine 90%+ is peered or hosted inside ISPs, not via transit How hosting inside ISPs function? Does ISP have to MITM? I heard similar claims for Netflix and other streaming media, like ISPs host/cache the data themselves. Do they have to have some agreement with Steam/Netflix?

Yea netflix will ship a server to an ISP (Cox, comcast, starlink, rogers, telus etc) so the customers of that ISP can access that server directly. It improves performance for those users and reduces the load on the ISP’s backbone/transit. Im guessing other large companies will do this as well.

A lot of people are using large distributed DNS servers like 8.8.8.8 or 1.1.1.1 and these cansometimes direct users to incorrect CDN servers, so EDNS was created to help with it. I always use 9.9.9.11 instead of 9.9.9.9 to hopefully help improve performance.

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#178

Post by fleabitdev »

pavel_lishin wrote: post #133
fleabitdev wrote: post #28

Back of the envelope, in the two years since the game was released, this single bug has wasted at least US$10,000,000 of hardware resources. That's a conservative estimate (20% of people who own the game keep it installed, the marginal cost of wasted SSD storage in a gaming PC is US$2.50 per TB per month, the install base grew linearly over time), so the true number is probably several times higher. In other words, t…

> the marginal cost of wasted SSD storage in a gaming PC is US$2.50 per TB per month Out of curiousity, how do you come up with a number for this? I would have zero idea of how to even start estimating such a thing, or even being able to tell you whether "marginal cost of wasted hard drive storage" is even a thing for consumers.

I'd be very interested in hearing alternative estimates, but here's my working:

The lowest cost I could find to rent a server SSD was US$5 per TB-month, and it's often much higher. If we assume that markets are efficient (or inefficient in a way that disadvantages gaming PCs), we could stop thinking there and just use US$2.50 as a conservative lower bound.

I checked the cost of buying a machine with a 2 TB rather than 1 TB SSD; it varied a lot by manufacturer, but it seemed to line up with $2.50 to $5 per TB-month on a two-to-five-year upgrade cycle.

One reason I halved the number is because some users (say, a teenager who only plays one game) might have lots of unused space in their SSD, so wasting that space doesn't directly cost them anything. However, unused storage costs money, and the "default" or "safe" size of the SSD in a gaming PC is mostly determined by the size of games - so install size bloat may explain why that "free" space was purchased in the first place.

> whether "marginal cost of wasted hard drive storage" is even a thing for consumers

As long as storage has a price, use of storage will have a price :-)

Re: Helldivers 2 devs slash install size from 154GB to 23GB

#180

Post by pdntspa »

snet0 wrote: post #16

> With their latest data measurements specific to the game, the developers have confirmed the small number of players (11% last week) using mechanical hard drives will witness mission load times increase by only a few seconds in worst cases. Additionally, the post reads, “the majority of the loading time in Helldivers 2 is due to level-generation rather than asset loading. This level generation happens in parallel wi…

I have heard that in many scenarios it is faster to load uncompressed assets directly rather than load+decompress. Load time is prioritized over hard drive space so you end up with the current situation.
Post reply on HN