Zsync: Differential file downloading over HTTP using the rsync algorithm (2010) • Hacker News Board

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#41

Post by dividuum » Tue, Oct 17, 2017, 7:09 PM UTC

alex_hirner wrote: post #33

dividuum wrote: post #20

Earlier quoted context omitted.

Not a major user, but we're using zsync for system updates of our Raspberry Pi based digital signage operating system ( https://info-beamer.com/hosted ). It's pretty great and offers a few things we couldn't do with bittorrent: Every time we have a new release we put together an install.zip file of everything required (kernel, firmware files, initrd, squashfs). Users can download this file directly and unzip it on th…

Did any IoT device management service fit the bill too (back then or now)? We are heading towards a similar use case.

OS development for info-beamer started in 2013. I'm fairly sure nothing even close was available back then. I'm not sure about today. So far I don't regret the NIH approach we took.

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#42

Post by eeZah7Ux » Tue, Oct 17, 2017, 8:45 PM UTC

iFire wrote: post #40

eeZah7Ux wrote: post #37

Earlier quoted context omitted.

How is the wharf protocol better than zsync?

https://itch.io/docs/wharf/design-goals.html Describes the goals. https://itch.io/docs/wharf/algorithms/diff.html and https://itch.io/docs/wharf/algorithms/apply.html describe patching The most important thing is that Itchio runs a business on the usability of this system.

Is it outperforming zsync in terms of bandwidth? That's the most important question for me.

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#43

Post by a4dev » Tue, Oct 17, 2017, 10:49 PM UTC

Lennart Poettering has an interesting tool, casync , which overlaps with zsync. Claims to be a better solution for image syncing. Reasons given here: http://0pointer.net/blog/casync-a-tool-for-distributing-file...

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#44

Post by deelowe » Wed, Oct 18, 2017, 12:05 AM UTC

oliwarner wrote: post #18

deelowe wrote: post #6

Earlier quoted context omitted.

It uses a special file that has to be installed on the server. Perhaps you thought (as did I) that this was supposed to work with any file over http? That does not appear to be the case. http://zsync.moria.org.uk/server

Installed on? Uploaded to. It's just a pair of files. The big thing you're trying to transfer and the zsync file that details the content of that first file, to guide the downloader.

I'm not sure I see the difference?

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#45

Post by js2 » Wed, Oct 18, 2017, 3:46 AM UTC

js2 wrote: post #9

2005. So I guess this didn't catch on.

Thank you to the sibling comments for mentioning that you're using zsync. I'm sorry my comment was so negative.

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#46

Post by DrJosiah » Wed, Oct 18, 2017, 4:22 AM UTC

deelowe wrote: post #44

oliwarner wrote: post #18

Earlier quoted context omitted.

Installed on? Uploaded to. It's just a pair of files. The big thing you're trying to transfer and the zsync file that details the content of that first file, to guide the downloader.

I'm not sure I see the difference?

You can upload the .zsync file to anywhere Apache, Nginx, etc., can read, and it all just works.

You don't need a plugin, you don't need to change your httpd.conf, ...

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#47

Post by IanCal » Wed, Oct 18, 2017, 7:47 AM UTC

rripken wrote: post #36

IanCal wrote: post #16

Earlier quoted context omitted.

Interesting note on the site is that these files don't need to be on the original server. Anyone can generate and host them. > in fact, the .zsync can be generated and offered by a third party, while still leaving most of the download to the original distribution site.

Interesting. How would that work for clients? If the destination file already exists on the client look for a .zsync on the hosting server, if its not there look for one at https://thirdpartyzsyncs.com?url=someurl ? What happens if the .zsync is out of sync with the resource?

The client, from my very limited testing, expects the url of the zsync file as an input. The zsync file can point to anywhere else for listing the canonical version.

So you'd zsync http://third-party/file.zsync and this file would contain the url for the main file.

> What happens if the .zsync is out of sync with the resource?

I'm not sure, I've not managed to get it working on a simple file yet.

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#48

Post by oliwarner » Wed, Oct 18, 2017, 1:28 PM UTC

deelowe wrote: post #44

oliwarner wrote: post #18

Earlier quoted context omitted.

Installed on? Uploaded to. It's just a pair of files. The big thing you're trying to transfer and the zsync file that details the content of that first file, to guide the downloader.

I'm not sure I see the difference?

Within the wider context of your comment, "install" implied a setup process or configuration change.

Re: Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)

#49

Post by deelowe » Wed, Oct 18, 2017, 2:41 PM UTC

DrJosiah wrote: post #46

deelowe wrote: post #44

Earlier quoted context omitted.

I'm not sure I see the difference?

You can upload the .zsync file to anywhere Apache, Nginx, etc., can read, and it all just works. You don't need a plugin, you don't need to change your httpd.conf, ...

Yeah. I assumed that.