Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Syncthing Usage Data

data.syncthing.net

Post reply on HN
Print view View original

41–50 of 125 posts

Re: Syncthing Usage Data

#41

Post by mduerksen »

Syncthing is sorely missing iOS support. I know they have their reasons not to implement it[0], but I feel that feature might be worth the hassle.

Photo sync on iOS without iCloud is a huge pain atm.

My backup strategy involves a NAS folder that stays up to date (i.e. also deletes photos that I delete on my phone), and which I occasionally sort into final location. After that I can free the space on my phone.

All other options (including Resilio Sync) are a add-only "sync"[1], which makes it annoying to sift through all the photos including those I already weeded out a week ago.

[0] https://docs.syncthing.net/users/faq.html#why-is-there-no-io...

[1] In theory, Resilio has a real sync, but for some reason, not for the iOS camera roll.

Re: Syncthing Usage Data

#42

Post by problems »

swiley wrote: post #39
problems wrote: post #35

Interesting that the mobile usage seems to be so low from this, I sync my desktop, NAS and phone using Syncthing constantly. I also reject any sort of statistics though, so I'm probably not helping.

I've run both the mobile and desktop version on my phone and it just drains the battery way too fast. I ended up just using an rsync cron job to sync my phone over ssh while everything else uses syncthing (and yes, I know this means I can't delete anything but I very rarely do anyway.)

You can set it to sync only on wifi and/or only when charging, which has always done the job for me. Depends on your use case, but things like password DBs need to be update-able from both directions, which rsync doesn't really do. On the other hand, I can rely on the fact that I don't need my home to be sync'd up unless I'm actually at home, on wifi.

Re: Syncthing Usage Data

#43

Post by zurn »

gravypod wrote: post #38

Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size. Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it. If this existed I'd have somethin…

Syncthing does redundancy by default, no? Just add a node.

But RAID and Syncthing aren't substitutes for backups, for the same reason: no restore if your data get hosed.

Re: Syncthing Usage Data

#44

Post by metalliqaz »

gravypod wrote: post #38

Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size. Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it. If this existed I'd have somethin…

This is an interesting idea, but I don't think it could work the way you are describing it (like RAID). Remember the limitations of RAID, and then add intermittent failures and huge latency to the communication between the "drives". It would not be able to remain coherent in real time.

Now, I think you could definitely set up a system that could handle the distributed data like this and allow you to access it, but I am skeptical that you could construct such a system that would be able to read/write/update/delete files on the fly and retain N-node failure tolerance at all times. You would probably have to allow the system to replicate data slowly.

There is also the problem of network splits. When they rejoin, how do they resolve conflicts and such?

Re: Syncthing Usage Data

#45

Post by gravypod »

zurn wrote: post #43
gravypod wrote: post #38

Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size. Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it. If this existed I'd have somethin…

Syncthing does redundancy by default, no? Just add a node. But RAID and Syncthing aren't substitutes for backups, for the same reason: no restore if your data get hosed.

I don't really want syncing. I want a distrabution of the blocks of the filesystem that allows for redundant storage & expansion of storage space.

RW times would be slower but storage would be bigger and more redundant.

Re: Syncthing Usage Data

#46

Post by icebraining »

gravypod wrote: post #38

Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size. Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it. If this existed I'd have somethin…

It's not exactly a filesystem, but git-annex allows you to configure multiple nodes, and to defined the minimum number of copies each file should have (globally or per file type). Using the daemon (assistant), the nodes will automatically copy files from other nodes until the number is reached.

Re: Syncthing Usage Data

#47

Post by brotherjerky »

microtonal wrote: post #31
okket wrote: post #15

Earlier quoted context omitted.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. E.g. you can set a syncthing instance as an 'introducer', so you only have to establish a connection to this device to get offered connections to all others devices. You still need to authorise these introductions on the devices, but you do not need to type in the crypto verification strings.

This is the price you have to pay for a (secure) decentralised system, but syncthing tries to help where it can. In general, Resilio Sync's 'key per folder' model seems to work better. I can give a key to other people and they can join the swarm without extra work on either sides. Also, it has a logical extension to encrypted-only peers: they get a derived key that can be used to sync in the swarm, but cannot decrypt…

Where can I read about the swarm capability? I'm searching for something like how BTSync worked, but I don't see how Syncthing does this. It seems like one to one sharing, per: https://forum.syncthing.net/t/can-i-use-synthing-to-distribu...

Re: Syncthing Usage Data

#48

Post by onli »

gravypod wrote: post #45
zurn wrote: post #43

Earlier quoted context omitted.

Syncthing does redundancy by default, no? Just add a node. But RAID and Syncthing aren't substitutes for backups, for the same reason: no restore if your data get hosed.

I don't really want syncing. I want a distrabution of the blocks of the filesystem that allows for redundant storage & expansion of storage space. RW times would be slower but storage would be bigger and more redundant.

Is it possible you look for something like Ceph? https://en.wikipedia.org/wiki/Ceph_(software)

Re: Syncthing Usage Data

#49

Post by gravypod »

metalliqaz wrote: post #44
gravypod wrote: post #38

Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size. Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it. If this existed I'd have somethin…

This is an interesting idea, but I don't think it could work the way you are describing it (like RAID). Remember the limitations of RAID, and then add intermittent failures and huge latency to the communication between the "drives". It would not be able to remain coherent in real time. Now, I think you could definitely set up a system that could handle the distributed data like this and allow you to access it, but I…

This system could largely be non-volatile. Each node would just be writing data like...

    _--_--.tar.xz
into the filesystem and another server would be reading through everything and processing it (not in real time, but as a batch system).

It's ok for things to be left over for collection at the next cycle.

Also netsplits aren't much of an issue if you're largely just doing writes and reads on seperate files. It only becomes a problem for short-term and currently processed files (that are being read and written to in real time).

This is more of a long-term cold storage system and not nodes talking to each other.

TL;DR

    "It's not a bug it's a feature"

    Patient: "Doctor, how do I stop the pain I get from doing this"
    Doctor: "You stop doing that"

Re: Syncthing Usage Data

#50

Post by newman314 »

vocatus_gate wrote: post #23
odabaxok wrote: post #12

Can anyone share their experiences with Syncthing compared to Resilio Sync (aka Bit Torrent Sync)? I am using Resilio Sync since its beta version, but they have added a lot of restrictions since then and made some functions only available for paying users. I am concerned about ease of use (setup and everyday usage) on Windows and Android.

Oh boy can I ever. We used Resilio Sync (aka Bittorrent Sync aka BT Sync) for a while for the Tron project ( https://reddit.com/r/TronScript ). It worked great for our use-case: distributing a large number of files to a large number of nodes who required read-only access. Anytime I changed a file on the master node it would blast out to everyone else while simultaneously preventing any other node in the swarm from pr…

Do you have a reference to this 32 node cap for Resilio? I just tried doing a quick search and did not see any specific results showing this limit. Thanks.
Post reply on HN