I don't see why any of this is needed. Just install Dropbox, and...
Let them use rsync for you?
How does rsync work?
31–40 of 54 posts
Re: How does rsync work?
#32This was a great write up. I've already sent it to a few people. On the question of what happens if a file's contents change after the initial checksum, the man page for rsync[0] has an interesting explanation of the *--checksum* option: > This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a "quick check" that (by default) checks if each fil…
> For protocol 30 and beyond (first supported in 3.0.0), the checksum used is MD5. For older protocols, the checksum used is MD4. Newer versions (≥3.2?) support xxHash and xxHash3: * https://github.com/WayneD/rsync/blob/master/checksum.c * https://github.com/Cyan4973/xxHash * https://news.ycombinator.com/item?id=19402602 (2019 XXH3 discussion)
I thought xxHash was only used for the chunk hash, not the whole file hash?
Re: How does rsync work?
#33I encountered a strange situation 2 days ago. I rsync my pdf files periodically between my harddrives. rsync showed no differences between two folder trees, but if I did `diff -r` between the two, 3 pdfs came out different. I checked the three individually but they showed no corruption or changes either side. How can this happen? Edit: the hard drive copy is previously rsynced from this copy & both copies are mirrore…
The same happend to me syncing to an SD card. The reason may be different timestamp resolutions, which make same files look different. I synced from HFS+ to FAT32 back then.
Re: How does rsync work?
#34Rsync worst issue is someone port scanning and brute force their way into your system. Turn off your port.
Most people use rsync over ssh
Re: How does rsync work?
#35Earlier quoted context omitted.
Unless you are also doing FS-level deduplication using the same checksums, it generally makes no sense for these to be cryptographic hashes, so they're not necessarily suitable for this purpose. IIRC neither ZFS nor btrfs use cryptographic hashes for checksumming by default.
> on is a short hand for fletcher4 for non-deduped datasets and sha256 for deduped datasets * https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Chec... * https://people.freebsd.org/~asomers/fletcher.pdf * https://en.wikipedia.org/wiki/Fletcher%27s_checksum Strangely enough SHA-512 is actually (50%) faster than -256: > ZFS actually uses a special version of SHA512 called SHA512t256, it uses a different initial v…
It seems that SHA512t256 is another name for SHA-512/256. It's a shame that the initialization is different from SHA-512, as it would have been very useful to be able to convert a SHA-512 hash into a SHA-512/256 hash.
Re: How does rsync work?
#36I don't see why any of this is needed. Just install Dropbox, and...
I'm gonna interpret this in the best faith possible, assume you're referencing the infamous Dropbox HN comment, upvote you to counteract the downvotes from people who missed the joke, and link to the aforementioned comment: https://news.ycombinator.com/item?id=9224
Re: How does rsync work?
#37Earlier quoted context omitted.
rsync uses a heuristic based on file times and sizes to compare files. to compare file content use the --checksum feature (computationally expensive to run)
Yes but it doesn't answer why rsync & checksum pass the set of files as same, but diff reports them different.
It isn't clear from the conversation so far: do you use "--checksum"?
Re: How does rsync work?
#38Rsync worst issue is someone port scanning and brute force their way into your system. Turn off your port.
Don't most consumer routers have all ports blocked? Who is connecting a computer directly to the modem these days?
Most of them do NAT, which isn't semantically the same as blocking all ports, but functionally it is.
Re: How does rsync work?
#39Nice write up. rsync is great as an application but I found it more cumbersome to use when wanting to integrate it into my own application. There's librsync but the documentation is threadbare and it requires an rsync server to run. I found bita/bitar ( https://github.com/oll3/bita ) which is inspired by rsync & family. It works more like zsync which leverages HTTP Range requests so it doesn't require anything runnin…
Always happy to see my pet project mentioned (bita) and that it is actually being used by others than me :)
Re: How does rsync work?
#40I don't see why any of this is needed. Just install Dropbox, and...
I'm gonna interpret this in the best faith possible, assume you're referencing the infamous Dropbox HN comment, upvote you to counteract the downvotes from people who missed the joke, and link to the aforementioned comment: https://news.ycombinator.com/item?id=9224
The irony is that BrandonM's classic HN comment makes more sense these days, as Dropbox continues its evolution towards Microsofthood. Increasingly, using Dropbox means putting up with an unceasing deluge of product promotions while you're trying to get your work done. No such annoyances with rsync and ftp and the like.
Just last week, Dropbox unilaterally decided I didn't want local copies of the shared files on my laptop, which made for some awkwardness inside a secured facility with no Internet access.