Live data from Hacker News

Unison File Synchronizer

cis.upenn.edu

51–60 of 102 posts

Re: Unison File Synchronizer

#51
post #49

Earlier quoted context omitted.

Unison suffers from the Lisp Curse - it's written in OCaml, a similarly powerful but small-audience language. It "just works", so there doesn't need to be much activity.

Absolutely, I have a entry in my journal about wanting to rewrite Unison in C. Never got around to it, but it would still be fun. The same goes with Wings 3D, cool app, but Erlang? Really?

oof. I think we've gotten much different takeaways from The Lisp Curse.

Re: Unison File Synchronizer

#52
post #49

Earlier quoted context omitted.

Absolutely, I have a entry in my journal about wanting to rewrite Unison in C. Never got around to it, but it would still be fun. The same goes with Wings 3D, cool app, but Erlang? Really?

oof. I think we've gotten much different takeaways from The Lisp Curse.

Maybe, I really like Lisp and especially Scheme. Don't get me wrong. I just think that a lot of projects suffer from being written in a language with a small community or even, at the time of conception, a thriving yet fleeting community (Perl, Ruby, etc).

Re: Unison File Synchronizer

#53
post #50

Is this project still going? I used to use it a lot for duplicating my project directories between my lap PC and my home PC. It was so powerful and so useful at the time. Honestly I thought it had died. I use Syncthing now. It's really useful.

I used Unison to sync between two laptops. In using only one laptop now and I switched to Syncthing to sync some folders of my phone, tablets and laptop. Unfortunately Unison doesn't run on Android. I liked the way it could perform a merge of the folders from different computers.

Can you not mount the android file system on your laptop, and sync the two different directory trees, running exclusively from the laptop?

Re: Unison File Synchronizer

#54
post #33

I can't praise Unison enough. I've been using it every day for over 10 years and is easily one of the most useful pieces of software I've ever come across. I've donated multiple times over the years.

"I can't praise Unison enough. I've been using it every day for over 10 years and is easily one of the most useful pieces of software I've ever come across." Unison was the first backup binary that we built into the rsync.net platform - breaking our original design goal of only offering client agnostic SSH and the tools that would run over that. Shortly afterward we also added rdiff-backup. Both of these tools were q…

> Can anyone guess how we do that?

Like you did with attic and borg? Quoting you on January 2016:

> We solved the problem by (cx)freezing the attic and borg python tools into binary executables. So, still no python in our environment (reducing attack surface) but the ability to run attic and borg just like they are meant to be run.

As seen here: https://news.ycombinator.com/item?id=10925123

Re: Unison File Synchronizer

#55
post #35

Earlier quoted context omitted.

Pretty hard. Unison is nice, but... - Unison is written in OCaml, which is (probably) a perfectly fine language but not commonly used - Unison synchronizes files to files, but for a Dropbox-like system you really want deduplication for space savings (i.e. server-side storage is a bunch of pointers to content-addressed blocks.) - in general, it's not clear to me that the client is really the hard part of Dropbox. Note…

> there's a million fiddly things to get right, and Unison hasn't had that much usage Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. Such models are also easily representated in OCaml; which I can assure you is more than a fine language, especially if one cares about correctness. Dropbox has struggled to get these de…

>Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness.

That's just about the sync process/stages, the easy part that can actually be formalized.

The "million fiddly things" are about OS and filesystem issues, incompatibilities, and so on, and Dropbox has a hugely larger test base for those things...

Re: Unison File Synchronizer

#56
post #55
post #35

Earlier quoted context omitted.

> there's a million fiddly things to get right, and Unison hasn't had that much usage Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. Such models are also easily representated in OCaml; which I can assure you is more than a fine language, especially if one cares about correctness. Dropbox has struggled to get these de…

> Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. That's just about the sync process/stages, the easy part that can actually be formalized. The "million fiddly things" are about OS and filesystem issues, incompatibilities, and so on, and Dropbox has a hugely larger test base for those things...

I disagree. POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs. My proof of this is the number of different and successful filesystems for Unix/Linux. If the abstraction didn't work, everyone would be forced to use the same filesystem.

The issues and subtleties are with bidirectional sync. It is not "the easy part". Dropbox didn't get it right in the past, we have no proof they have it right now.

Re: Unison File Synchronizer

#57
post #33

Earlier quoted context omitted.

"I can't praise Unison enough. I've been using it every day for over 10 years and is easily one of the most useful pieces of software I've ever come across." Unison was the first backup binary that we built into the rsync.net platform - breaking our original design goal of only offering client agnostic SSH and the tools that would run over that. Shortly afterward we also added rdiff-backup. Both of these tools were q…

> Can anyone guess how we do that? Like you did with attic and borg? Quoting you on January 2016: > We solved the problem by (cx)freezing the attic and borg python tools into binary executables. So, still no python in our environment (reducing attack surface) but the ability to run attic and borg just like they are meant to be run. As seen here: https://news.ycombinator.com/item?id=10925123

Correct!

Re: Unison File Synchronizer

#58
post #38

Is this project still going? I used to use it a lot for duplicating my project directories between my lap PC and my home PC. It was so powerful and so useful at the time. Honestly I thought it had died. I use Syncthing now. It's really useful.

I used to use it daily to sync projects, music, and documents between computers too. It was really nice once I got it configured to my liking. I would sync my work computer to a external hard drive and the external drive to my home computer, thus creating a very powerful sneakernet and an implicit backup on the external. I reluctantly installed Dropbox one day after forgetting my external drive at home. Honestly, tha…

Sorry, I'm probably missing something obvious (and am asking out of genuine curiosity here), but why didn't you go back to using Unison? And what are you using now that you quit Dropbox?

Re: Unison File Synchronizer

#59
post #56
post #55

Earlier quoted context omitted.

> Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. That's just about the sync process/stages, the easy part that can actually be formalized. The "million fiddly things" are about OS and filesystem issues, incompatibilities, and so on, and Dropbox has a hugely larger test base for those things...

I disagree. POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs. My proof of this is the number of different and successful filesystems for Unix/Linux. If the abstraction didn't work, everyone would be forced to use the same filesystem. The issues and subtleties are with bidirectional sync. It is not "the easy part". Dropbox didn't get it right in the past, we have no…

> POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs

I know nothing about POSIX, but I presume we all want stronger guarantees to avoid the risk of syncing a corrupted file.

https://danluu.com/file-consistency/

Re: Unison File Synchronizer

#60
post #59
post #56

Earlier quoted context omitted.

I disagree. POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs. My proof of this is the number of different and successful filesystems for Unix/Linux. If the abstraction didn't work, everyone would be forced to use the same filesystem. The issues and subtleties are with bidirectional sync. It is not "the easy part". Dropbox didn't get it right in the past, we have no…

> POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs I know nothing about POSIX, but I presume we all want stronger guarantees to avoid the risk of syncing a corrupted file. https://danluu.com/file-consistency/

That's really the job of the filesystem to protect against corruption using checksums. ZFS will do this and hopefully more will follow. I suppose a file sync tool could detect a change of contents with no change of mtime, but that would be expensive.
Post reply on HN