Mysteries of Dropbox: Testing of a Distributed Sync Service (2016) [pdf]

There was a discussion of a self-built dropbox on the frontpage (https://news.ycombinator.com/item?id=47673394). This is just to show that dropbox is thoroughly tested for all kinds of wierd interactions and behaviours across OS using a very formal testing framework.

Can you share which difficult and non-intuitive corner cases there are? I guess debouncing, etc.

So from what I am seeing in this with a brief look over it, the only cases in which data loss seemed to occur were when two clients were editing the same file temporally close to each other? I.e. you end up creating something similar to a git merge conflict, which cannot be solved automatically well, and thus can generate loss of data.

Anything written by John Hughes is worth a read. He also also wrote quickcheck.

The way I used to explain it:

Imagine that you are on a plane, (and don't have an internet connection). You edit a file.

At the same time, I edit that file.

What should we do? We can't possibly know every file format out there, and implement operational transform for all of them.

Now, imagine that we both edit the same file, at the same instant. One of us is going to submit the change first, and the other will submit it second. It's the same use case, and there's no way to avoid this.

---

Renaming folders was a lot weirder, because you get situations like:

I rename a folder, but you save a new change to a file in that renamed folder and your computer doesn't know about the renamed folder.

Or, I rename a folder and you have a file open. That application has an open file handle to that file, so we can't just rename the folder. What do we do? (This is how Excel does it.)

Or, I rename a folder and you have a file open, but that application doesn't have an open file handle to that file. What happens when you try to save the file and it's been moved? (This is how most applications do it.)

---

Application bundles (on Mac) were weird because we didn't support the metadata needed to sync them.

---

The general "Merge" use case, which had to do with the fact that Syncplicity could sync folders anywhere on disk. (As opposed to the way That Dropbox, Google Drive, and OneDrive stick everything into a single folder.) We'd have customers disconnect a folder, and then re-add it to the same location. The problem was if they were disconnected for a long time, they would "merge" the old version of the folder into the new one:

If you edited a file while disconnected, it hit the same "multiple editors" use case that I mentioned above.

If someone deleted a file but you still had it, we'd recreate it. (We can't read minds, you know!)

If someone renamed a folder, but you still had the old path, we'd re-add it.

I remember overhearing non-programmer product managers trying to talk through these use cases and just getting overwhelmed with the complexity and realizing they were deep, deep over their heads.

---

A lot of these corner cases were smoothed over when we wrote "SyncDrive", which was a virtual disk drive, because all of the IO came through us. (Instead of scanning a folder to understand what the user did.)