I think we should remember the context here.
They're using the outdated stingray engine and this engine is designed for the days of single or dual core computers with spinning disks. They developed their game with this target in mind.
Mind you, spinning disks are not only a lot more rare today but also much faster than when Stingray 1.0 was released. Something like 3-4x faster.
The game was never a loading hog and I imagine by the time they launched and realized how big this install would be, the technical debt was too much. The monetary cost of labor hours to undo this must have been significant, so they took the financial decision of "We'll keep getting away with it until we can't."
The community finally got fed up. The steamdb chart keeps inching lower and lower and I think they finally got worried about permanently losing players that they conceded and did this hoping to get those players back and to avoid a further exodus.
And lets say this game is now much worse on spinning disk. At the end of the day AH will choose profit. If they lose that 10% spinning disk people who wont tolerate the few seconds change, the game will please the other players, thus making sure its lives on.
Lastly, this is how its delivered on console, many of which use spinning media. So its hard to see this as problematic. I'm guessing for console MS and Sony said no to a 150gb install so AG was invested in keeping it small. They were forced to redo the game for console without this extra data. The time and money there was worth it for them. For PC, there's no one to say no, so they did the cheapest thing they could until they no longer could.
This is one of the downsides of open platforms. There's no 'parent' to yell at you, so you do what you want. Its the classic walled garden vs open bazaar type thing.