I wonder where they're hosting this, since most cloud providers would cost a fortune to download all the data.
I don’t think they are hosting it? Seems like client side cache layer calling steam DL APIs?
Are you asking if that's what they are doing?
Solving the 100GB Game Download
71–80 of 108 posts
Re: Solving the 100GB Game Download
#72This seems an incredible technical accomplishment. My one question is, is it supposed to be solving for disk space constraints or connection speed constraints? I'm guessing both, but the former is pretty cheaply solved these days with a new drive (or by finally admitting you only need to have 2-3 games installed at once and probably aren't playing a dozen games in parallel), and I'm not sure it seems to be solving th…
If you have large drives then you can have best of both worlds where it downloads and caches your most used stuff without you having to wait for a full download to start playing, and then next time you play it's already downloaded.
Re: Solving the 100GB Game Download
#73I wonder where they're hosting this, since most cloud providers would cost a fortune to download all the data.
Sounds like he is running a steam cache out of Chicago and Germany so likely running it off of a VPS service for now.
Honestly if he gets this working really well (especially on linux / steamdeck) this could be an excellent aquire + hire by valve.
Re: Solving the 100GB Game Download
#74Earlier quoted context omitted.
That sounds like a lot of engineering effort for something with minimal payoff. Carefully balancing early vs late offsets for the one time payoff of playing while the game is still downloading. So many potential failure states vs just saying, "Wait another hour and then start"
To you it might sound like a lot of effort but for a user who is on his 1 day off for the week not having to wait twice the amount of time to log in to play his game probably means more than you or I could ever imagine. Blizzard used to be about making everyone have an excellent experience and some of the legacy features they've had for years seriously still do show what kind of commitments to that fact they have mad…
Blizzards downloaded is top tier. The startup time from initial, bare bones install to playable is very fast.
It starts with placeholder content to get you running, and anything immediately necessary for your area. For example, if you’re in a capital city, it doesn’t wait to download the entire city, just where you are. It can rough out the other areas and add details almost in real time as you start making your way through it.
Things get more difficult if you mount up and start flying more world to load, but even still it doesn’t load it all.
There’s certainly a cost, there can be FPS loss during busy areas, but by the time you get to something important, it should all be loaded.
Contrived example is you’re in a raid, some reason you feel you need to reinstall, you could do that, fire up the game, it’ll load the raid and by the time you hit the first boss, it’ll all be there. This will take minutes.
Since I leave the client running, it updates in the background. During new releases it starts to download the entirety of the new release a couple of weeks before it’s due, to get the bulk of the content down. When they cut the release, it’s typically a much smaller update to turn the new content on. I rarely have to wait to play the game.
Experiencing a few other games makes me really appreciate the work Blizzard had done in this regard. It’s top drawer.
Re: Solving the 100GB Game Download
#75I like it, but it does not "solve" the big downloads issue. You know what would solve the 100 GB game download problem? Reducing that to 30 or 40 GB, not inventing an (admittedly clever and nice) man in the middle streaming things to you in byte-sized chunks.
Virtual texturing ( or people may have heard of MegaTextue ) meant we could easily scale 100s GB for high quality Graphics. We are already quite CPU bound in the PS5 era sp I dont see super-compression to happen. The only way to solve this would be something like Generative Graphics. But I think that is an worst option than large storage space.
Re: Solving the 100GB Game Download
#76Slightly offtopic, is there a way to setup a virtual file system that caches most used files and removes least used of a project in internal SSD from an external HDD?
i recommend primocache on Windows
seconded. Primocache is wonderful, and doesn't suffer from the glaring issue using the powershell hack of Windows Storage Spaces [1]. You can set this up, with proper powershell scripts/commands, but Xbox games won't install to a Storage Space. It boggled my mind when I finally tracked this down.
Primo Cache [2] has one disadvantage over storage spaces - the SSD cache space is dedicated as Cache, and does not add to the total space of the drive. 1TB SSD + 4TB hdd = 4TB available, vs storage spaces which will give you 5TB. This minor (to me) downside is outweighed by the fact it works for everything. It shows up as a normal drive, not a virtual disk, and it just works. I recommend it highly for anyone who has limited SSD space.
1: https://github.com/freemansoft/win10-storage-spaces 2: https://www.romexsoftware.com/en-us/primo-cache/
Re: Solving the 100GB Game Download
#77Earlier quoted context omitted.
That sounds like a lot of engineering effort for something with minimal payoff. Carefully balancing early vs late offsets for the one time payoff of playing while the game is still downloading. So many potential failure states vs just saying, "Wait another hour and then start"
To you it might sound like a lot of effort but for a user who is on his 1 day off for the week not having to wait twice the amount of time to log in to play his game probably means more than you or I could ever imagine. Blizzard used to be about making everyone have an excellent experience and some of the legacy features they've had for years seriously still do show what kind of commitments to that fact they have mad…
Steam supports this as well but actually I have read that it is a lot of effort.
Firstly, your entire asset pipeline which combines and compresses resources needs to be aware of it.
Secondly, it's work that can't be reused cross platform. (The effort for Steam will not apply to PS5&etc).
Thirdly, it's another configuration to test, and as the user will forget about the feature after they get started playing, it's natural to just say it's too much effort.
Re: Solving the 100GB Game Download
#78Slightly offtopic, is there a way to setup a virtual file system that caches most used files and removes least used of a project in internal SSD from an external HDD?
On Linux you have bcache and flashcache, but they work on the block layer and not file layer.
And I believe the work-in-progress bcachefs also supports this use case.
Re: Solving the 100GB Game Download
#79I like it, but it does not "solve" the big downloads issue. You know what would solve the 100 GB game download problem? Reducing that to 30 or 40 GB, not inventing an (admittedly clever and nice) man in the middle streaming things to you in byte-sized chunks.
Agreed.
I am not condoning piracy, but when looking at game sizes, go check out fitgirl repacks. Right now there is a Steam game there reduced from 20GB to 10GB. Another is reduced from 3.5GB to 1.1GB. The downloads from there are consistently and significantly reduced.
If a pirate can do that, why not a game company? or Steam?
Re: Solving the 100GB Game Download
#80I felt the urge to play Deadfire today so I got my gaming PC out of the closet. Shortly thereafter it went down a flight of stairs (don’t ask) and lost both wifi antennas. Downloading 20GB at about 5mbit was painfully retro, but I got the idea to Ethernet it to my MacBook and in a few clicks was borrowing its quite decent wifi connection. But my MacBook doesn’t have an Ethernet port or a normal usb port. So I got a d…
Obviously I don't know your setup, but that sounds harder than "move the computer next to the router, plug it in."
That's much less satisfying though.
The excitement when you revive an old device by using a USB stick to pass a file to Bluetooth and then send the file via infrared can't be beat.