Live data from Hacker News

Syncthing Usage Data

data.syncthing.net

121–125 of 125 posts

Re: Syncthing Usage Data

#121
post #15

Earlier quoted context omitted.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. E.g. you can set a syncthing instance as an 'introducer', so you only have to establish a connection to this device to get offered connections to all others devices. You still need to authorise these introductions on the devices, but you do not need to type in the crypto verification strings.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. In general, Resilio Sync's 'key per folder' model seems to work better. I can give a key to other people and they can join the swarm without extra work on either sides. Also, it has a logical extension to encrypted-only peers: they get a derived key that can be used to sync in the swarm, but cannot decrypt…

The model has been considered for Syncthing many times, but it boils down to the fact that:

1) There's no way to revoke access, and 2) Once the secret is out, it's out: you have to scrap it and start again from scratch.

Re: Syncthing Usage Data

#122

Earlier quoted context omitted.

Oh boy can I ever. We used Resilio Sync (aka Bittorrent Sync aka BT Sync) for a while for the Tron project ( https://reddit.com/r/TronScript ). It worked great for our use-case: distributing a large number of files to a large number of nodes who required read-only access. Anytime I changed a file on the master node it would blast out to everyone else while simultaneously preventing any other node in the swarm from pr…

That's interesting... so even the original 'master' client is treated equally? Other clients can decide to send/receive amongst themselves? How do you deal with that?

I basically post a warning that says "be aware that someone might attempt this." There's unfortunately nothing I can do about it with the way Syncthing is currently architected. We haven't had any malicious users yet but I'm sure we will eventually.

Fallback is just to provide a static .exe on the mirrors and leave the Resilio Sync node running alongside it.

Re: Syncthing Usage Data

#123
post #75
post #2

It amazes me that Syncthing handles rapidly-changing files better for me than Dropbox. Where Dropbox has left different versions of files on different computers even after they have changed multiple times, Syncthing happily synchronizes them without issues. And the fact that Syncthing is open source and allows me to keep files on my own machines is a big plus. However, I would love to see a paid option for easy backu…

Just a heads up: that almost certainly means SyncThing is failing to recognize some classes of conflicts and concurrency races, and is destroying some changes. Doing this right is a hard problem. I know they can be a pain to deal with, but we don't spin out conflicts for no reason! Disclosure: I work on the Dropbox sync engine.

I think that Dropbox typically handles conflicts quite well, and the issues I had are more likely bugs outside the conflict resolution implementation. I was a bit brief in my comment above, so let me elaborate in case you or someone else is interested:

The issues I had didn't result in conflicted files. Rather, after making a big change (i.e. switching git branches) some files were never updated or synced. Dropbox stopped picking up changes in the folder and eventually removed new changes once restarted.

The order of events were something along the lines of:

1) Did work on computer A that caused massive file changes (i.e. moving between git branches). 2) Moved to computer B to continue work. 3) Noticed files were old or missing on B. 4) Syncing files in some other folders worked, but nothing happened in the folder with missing files. 5) Restarted Dropbox on both machines in hope that this would trigger a fresh sync. 6) Observed files being reverted to old versions or deleted on machine A.

The end result was that Dropbox threw away the changes I had made on A and left me with the original state of B. I was able to recover the changes from a backup, so it was no big deal in the end (although it left me a bit scared I could have lost those files without noticing).

I was in contact with Dropbox support about the issue and explained in detail what I had done and what happened. I was offered help to recover the files, but since I had already done so, I just told them I didn't need any more support on the issue. I thought it might be because /proc/sys/fs/inotify/max_user_watches had a low value on one machine, so I wrote back that they might want to add back the old warning about this. However, the same problem with deleted files happened again after I had verified that this value was high enough on all machines.

I have also seen how a script run by a colleague managed to confuse Dropbox. The script was running a test which repeatedly created and deleted the same file before checking its correctness. Running the script in the Dropbox folder left him with some old version of this file and a failed test. Running the scirpt in a folder outside Dropbox left him with the correct final version of the file. He was only working on one machine.

And yes, I know it's "bad" to run scripts like this or switch git branches on top of sync software, but it happens, and it is interesting to see how different software handles these cases.

It should be noted that Dropbox usually handles these massive file changes well, so moving to Syncthing has for me been more about it being open source and the possibility to keep files on my own machines. I was just glad to see that Syncthing also handles heavy use cases gracefully.

Re: Syncthing Usage Data

#124

Earlier quoted context omitted.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. In general, Resilio Sync's 'key per folder' model seems to work better. I can give a key to other people and they can join the swarm without extra work on either sides. Also, it has a logical extension to encrypted-only peers: they get a derived key that can be used to sync in the swarm, but cannot decrypt…

The model has been considered for Syncthing many times, but it boils down to the fact that: 1) There's no way to revoke access, and 2) Once the secret is out, it's out: you have to scrap it and start again from scratch.

What is the problem with that?

You can always close the folder and start a new one.

Re: Syncthing Usage Data

#125

Earlier quoted context omitted.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. In general, Resilio Sync's 'key per folder' model seems to work better. I can give a key to other people and they can join the swarm without extra work on either sides. Also, it has a logical extension to encrypted-only peers: they get a derived key that can be used to sync in the swarm, but cannot decrypt…

Yeah, the only problem is you can never "un-give" a key, which means you either have to rebuild your cluster under a different key (having to re-give key to everyone else apart from that one guy you want to kick out), or deal with the person having access to the data forever.

If the data is encrypted using a session key and then the session key encrypted with a master key and stored with the data (concat them together) then it can. You can't revoke access to the data they've already got, of course, but as changes/additions are made they lose the ability to decrypt the new data because you've used a new session and master key.

Arq uses this method when encrypting blobs.

Post reply on HN