Good article. As a note, I love how he uses hand drawn diagrams. I have yet to find any tool that allow me to draw diagrams as fast as I can do it on a piece of paper.
Efficient game updates
11–20 of 27 posts
Re: Efficient game updates
#12Good article. As a note, I love how he uses hand drawn diagrams. I have yet to find any tool that allow me to draw diagrams as fast as I can do it on a piece of paper.
A Wacom tablet?
Re: Efficient game updates
#13Also relevant is Google's Courgette algorithm which is used to update Google Chrome - https://dev.chromium.org/developers/design-documents/softwar... which is stated to be much better than bsdiff which is what itch's is using here.
I suspect that it is hard to apply to other scenarios. It is much more specialized than a simple binary diff.
Re: Efficient game updates
#14Good article. As a note, I love how he uses hand drawn diagrams. I have yet to find any tool that allow me to draw diagrams as fast as I can do it on a piece of paper.
A Wacom tablet?
I have a Surface, it works reasonably well for the same task. But the same applies - it is not "pretty" svg graphics, but faster and easier. I suppose you could always make the initial diagram this way, and create it in a 2d cad like program if you need to.
Re: Efficient game updates
#15The rsync example confuses me a little bit. If you add a single bit to the front, then all the bytes are shifted into different blocks and nearly none will hash to match. But if you add a single bit, rsync still performs well. Can someone explain why that difference from the explanation? The problem also applies to the binary delta. Adding a prefix will shift everything forward causing a diff in everything. Bsdiff so…
The rsync algorithm divides the file into fixed size blocks only on the sending side, then calculates checksums for all blocks. On the receiving side, it tries to match them at all offsets, not just multiples of the block size.
Thus, in your example, the first (and possibly the last) block won't be found, but all other blocks will be found, shifted by an offset of 1.
Re: Efficient game updates
#16Earlier quoted context omitted.
A Wacom tablet?
I have a Surface, it works reasonably well for the same task. But the same applies - it is not "pretty" svg graphics, but faster and easier. I suppose you could always make the initial diagram this way, and create it in a 2d cad like program if you need to.
Does it have some sort of grid-layout so you can just draw linear sections etc.?
Is there any other case where prettification works better? (Splines, I suppose).
Re: Efficient game updates
#17The rsync example confuses me a little bit. If you add a single bit to the front, then all the bytes are shifted into different blocks and nearly none will hash to match. But if you add a single bit, rsync still performs well. Can someone explain why that difference from the explanation? The problem also applies to the binary delta. Adding a prefix will shift everything forward causing a diff in everything. Bsdiff so…
The rsync algorithm divides the file into fixed size blocks only on the sending side, then calculates checksums for all blocks. On the receiving side, it tries to match them at all offsets, not just multiples of the block size. Thus, in your example, the first (and possibly the last) block won't be found, but all other blocks will be found, shifted by an offset of 1.
Ah, seems obvious now that you say it. Thanks!
Re: Efficient game updates
#18Earlier quoted context omitted.
I have a Surface, it works reasonably well for the same task. But the same applies - it is not "pretty" svg graphics, but faster and easier. I suppose you could always make the initial diagram this way, and create it in a 2d cad like program if you need to.
Does it have some sort of grid-layout so you can just draw linear sections etc.? Is there any other case where prettification works better? (Splines, I suppose).
I haven't really searched for a program like that, and rather used it as a an electronic piece of paper.
Re: Efficient game updates
#19Interesting, but I'd like more details on what's happening at the client.
Take Steam for example. For some games, downloading the update takes seconds, but calculating diffs and extracting takes 10-20 minutes. That's great for Valve, because little bandwidth is used, but terrible at the client side. On top of that, the update process slows the rest of the system almost to a halt, because of all the hard drive activity.
I can potentially see this mechanism making the same mistake.
Re: Efficient game updates
#20Interesting, but I'd like more details on what's happening at the client. Take Steam for example. For some games, downloading the update takes seconds, but calculating diffs and extracting takes 10-20 minutes. That's great for Valve, because little bandwidth is used, but terrible at the client side. On top of that, the update process slows the rest of the system almost to a halt, because of all the hard drive activit…
> On top of that, the update process slows the rest of the system almost to a halt, because of all the hard drive activity
As far as I'm aware, that's a problem only on Linux, because Windows has a desktop-grade IO scheduler tuned to interactive usage (whereas in Linux both the CPU and IO schedulers are written for maximum throughput).