Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster LZ is not the answer to 150-250 GB video game downloads

richg42.blogspot.com

Post reply on HN
Print view View original

111–120 of 267 posts

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#111

Post by Sohcahtoa82 »

jjoonathan wrote: post #74
Waterluvian wrote: post #67

Earlier quoted context omitted.

In terms of attracting engineering talent, how does the video games industry fare? In a crunch situation, I can think of how easy it is for some to be able to circumnavigate various pitfalls while others fall into them and have no time to address it. Very much a skill/experience thing. There’s countless examples of games that run horribly given their level of presentation, or have no business being so impressively pe…

It attracts truly extraordinary talent. One of many examples: https://www.youtube.com/watch?v=eviSykqSUUw . There is more technical ambition in a single slide of that project than an entire department at places I've worked. However, it also tends to demand more and pay less, so there is churn and burnout and when it comes time to triage, the triage can cut very deep. If I were to dive into the JIRA for any one of the…

> However, it also tends to demand more and pay less, so there is churn and burnout and when it comes time to triage, the triage can cut very deep.

Yeah, I've often heard that the advice for people who really want to make games is to get some friends together and try to put out something together as a side gig while you work somewhere boring to pay the bills.

SWE at a game studio, especially any AAA game studio, is an awful experience. The expectations are high and the salaries are low because there's a seemingly unlimited bunch of young adults that want to make games.

And yes, it takes extraordinary talent. Your code needs to be able to generate a frame in under 16 milliseconds in order to maintain 60 fps. If you want to appease the hardcore players with these crazy 360 hz monitors, you need to get your game loop down to a mere 3 ms.

The amount of shortcuts and optimizations needed to pull that off is insane. At that level, you're trying to figure out how to minimize cache misses and branch mispredictions. How many Node devs are thinking about those?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#112

Post by Sohcahtoa82 »

KMnO4 wrote: post #10

Games want to be more photorealistic and the hardware to fake it is getting faster and cheaper. I remember in 2008 when Intel demoed ray tracing[0] for Quake Wars. It ran on a quad-socket (4 CPUs x 2.7GHz) at around 25 FPS, and even Intel admitted that ray tracing is just "brute forcing" the complex optical physics of a scene. Considering between then and now, we have much prettier looking games that do NOT require r…

> clever" techniques and algorithms

Lately I've become hyper-aware of games doing screen-space reflections and suddenly finding it distracting when I point my view down and the reflection of the scene at the horizon vanishes off the water/floor.

I upgraded from a 1070 to a 3080 right when the 3080 came out, and I actually have yet to play an RTX game.

> bloom

I hate that this ever became a thing.

Bloom just makes the scene blurry and bright parts end up blown out.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#113

Post by PeterisP »

Waterluvian wrote: post #67
terafo wrote: post #5

He completely misses the point. Games don't take hundreds of gigs because they have sub-optimal compression. They take hundreds of gigs because they don't use ANY compression. Jedi Survivor takes 139 gigs on Series X and mere 44 gigs on Series S. It didn't come out yet, but I'm 99.99% certain that this difference is just from inclusion of 8k textures into Series X version. That kind of scaling means that they don't u…

In terms of attracting engineering talent, how does the video games industry fare? In a crunch situation, I can think of how easy it is for some to be able to circumnavigate various pitfalls while others fall into them and have no time to address it. Very much a skill/experience thing. There’s countless examples of games that run horribly given their level of presentation, or have no business being so impressively pe…

They can get great people but then choose to spend their effort solely on features and ignoring things like size and speed - for example, the GTA V loading time issue (https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...) is a good example, where GTA V did many things really well technically but then completely ignored a huge loading performance issue for many years; things like that aren't "very much a skill/experience thing", the difference is whether the company cares.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#114

Post by Sohcahtoa82 »

tmilard wrote: post #102
tmilard wrote: post #97

Earlier quoted context omitted.

I it is true : I was very agressive on this one. But still, thousands and tousands of triangle feels like so much overweight for me. Specialy for indoor scenes. Quality for the eye is a lot to do with texture, NOT the number of triangle.

And I do not even talk about the GPU work to handle those thousands of triangles. With 300 points, my scene runs quick&fast even on budget Smartphones ! This is an important issues I can tell you. A video game that runs on all devices ...

At 300 points, a computer from 1996 could run your game.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#115

Post by haswell »

kevingadd wrote: post #94
hellotomyrars wrote: post #8

Game size is a perennial talking point and some games earn it and some games don’t. There was a phase of games shipping with uncompressed audio in every language they support for no good reason that ballooned file size (Titanfall) Some games have gotten around this problem by having higher resolution textures or additional languages being optional installs within whatever ecosystems DLC implementation is. I think thi…

Titanfall shipped uncompressed audio on purpose to cut out audio decompression CPU usage. You can disagree with the decision, but it wasn't an accident or negligence.

I realize they’re trying to optimize for a wide base, but I have plenty of spare cores to decompress audio, and SSD space is limited.

For me, the net result is that Titanfall was one of the first games to go when I needed space. Low hanging fruit.

Even if it was deliberate, I have to question the logic and wonder how much usage they enabled vs. uninstalls from file size over time.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#116

Post by ByThyGrace »

blibble wrote: post #58
overthrow wrote: post #20

Earlier quoted context omitted.

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

if it's on a modern SSD you can DMA it directly into the GPU no CPU required

Interesting. How modern are we talking here? Is this a "devil's in the details" kind-of feature?

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#117

Post by falcolas »

babypuncher wrote: post #30
overthrow wrote: post #20

Earlier quoted context omitted.

If you compress data, it will be smaller and require less i/o to load, so it will load faster. That's ESPECIALLY true on HDDs, but even on SSDs there are decompression algos that run faster than SSD bandwidth. The uncompressed asset cargo cult is hurting everyone who plays their games, no matter what kind of disk they use.

Jedi Survivor isn't shipping on any platforms constrained by an HDD, so it would be odd if they were making any optimizations for spinning rust, especially when they come at the cost of precious SSD space.

It’s releasing on PC, which could be constrained. It wont be for a majority of players, but HDD setups still exist.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#118

Post by kevingadd »

tmilard wrote: post #108
kevingadd wrote: post #99

Earlier quoted context omitted.

PNG decompression is way too slow. Some games use it but it's on the way out, either being replaced by codecs like Basis or QOI in most cases

Really ? It does not feel this way on my scene that has more than 90 .png files used for texture. The time to : download the globel zip file + open and reading the points and all textures => 5 to 7 seconds. This is super quick !

Switching from PNG to basis for my game (currently with a pretty minimal set of textures) was like a 50% reduction in load times with no other changes.

Note that I'm not storing assets at rest in a ZIP file because there's no reason to. If I was doing that it might close the gap some.

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#119

Post by kevingadd »

terafo wrote: post #107
kevingadd wrote: post #92

Earlier quoted context omitted.

You're saying Jedi Survivor doesn't use block texture compression? On which target? All targets? Can you capture a trace in RenderDoc to prove this? Or show raw bmp/uncompressed dds files on disk?

By texture compression I meant something like LZ4 or Kraken on top of block texture compression, sorry for ambiguity. Of course they use block level compression. But it seems that it's the only thing they use, as I said in my top level comment.

You'll find that's pretty common since block compression already reduces sizes pretty effectively. At install time the files will be compressed further over the wire before being decompressed, so I imagine the payoff is not that big in many cases. I do wonder how much of it is that a Kraken license costs money and it takes time to integrate, though...

Re: Faster LZ is not the answer to 150-250 GB video game downloads

#120

Post by Sohcahtoa82 »

LinuxBender wrote: post #11

I recall a lot of hype more than a decade ago about all the textures going away and being replaced with math formulas to generate everything on the fly. Games were supposed to get dramatically smaller. What ever happened to that?

> all the textures going away and being replaced with math formulas to generate everything on the fly.

Anyone who thought this was incredibly naive.

I look around my room and try to think about what textures could be generated procedurally. My walls, carpet, and ceiling? Yeah, sure. The stickers on my computer? No way.

Post reply on HN