341–350 of 363 posts
Magic wormhole - it traverses NATs, is encrypted, requires configuration of the source or destination. https://github.com/warner/magic-wormhole
Do you mean requires no configuration on the source or destination? :)
magic-wormhole: https://magic-wormhole.readthedocs.io/en/latest/ pipx install magic-wormhole wormhole send /some/big/stupid/thing.tar.gz
Earlier quoted context omitted.
For some reason I haven’t understood, syncthing would often miss transferring files but mark the entire transfer/sync as complete. No errors in the logs either. I tried it for a few months earlier this year, then gave up and switched to plain rsync in a shell. Based on that sample of one personal experience, I wouldn’t recommend syncthing to anyone. Note: I was using syncthing for a one way sync all the time, with se…
I've been using Syncthing extensively for years and never observed this behavior - it always either completes the transfer or fails to complete the transfer (which is obvious in the UI). A silent failure is certainly not typical. This sounds like a pretty severe bug, so if you still have the systems you were able to produce the behavior on it might be helpful to reproduce it and submit an issue.
I self-host a Nextcloud instance. If it's too big for a browser download, Bittorrent is the way to go.
This is interesting. What conditions would make a file too big for a browser download? If the server supports range requests (I'm almost certain Nextcloud does), then the browser can download in chunks and just retry any failed chunks.
Earlier quoted context omitted.
I have considered it. The one thing about Syncthing is that there is no true readonly way to sync. For example, someone may accidentally extract an archive in the Syncthing folder and it will sync everywhere. If Syncthing gets this feature I think I would be 100% onboard with it for LAN parties. Resilio I believe supports it but I would prefer an opensource alternative. https://github.com/syncthing/syncthing/issues/6…
You can use the send only feature in Syncthing to make your folder read only. https://docs.syncthing.net/users/foldertypes.html#folder-sen...
Earlier quoted context omitted.
What's an example of the syntax of this? I remember seeing a graybeard do something like this to huck some files to another machine.
Paraphrased from a real example: tar cf - files morefiles | ssh user@dest "tar xf -" If you want to insert, say, compression: tar cf - files morefiles | zstd | ssh user@dest "zstd -d | tar xf -"
Earlier quoted context omitted.
This is interesting. What conditions would make a file too big for a browser download? If the server supports range requests (I'm almost certain Nextcloud does), then the browser can download in chunks and just retry any failed chunks.
It's technically possible, but most browsers don't auto-resume, and most webservers will timeout a request after a certain amount of time has elapsed. You can get around all this with browser extensions, but it's honestly easier to use a more robust protocol (this is in the context of, say, a 10TB file that'll take you a few days to download).
5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination 5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination 10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! https://www.tidbitsfortechs.com/2013/09/never-underestimate-... And since you now buy 1TB micro SD cards, so perhaps I'd split th…
> Never underestimate the bandwidth of a station wagon full of backup tapes! The exact quote [0] by Andrew Tanenbaum is: "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.". Obligatory xkcd [1]. I met Andy Tanenbaum in 2008 at Vrije Universiteit in Amsterdam. I was the Technology Evangelist for AWS in Europe, giving a seminar there on cloud computing. I loved him. Funny, sm…
The data transfer rate, however, scales linearly with cardinality...
Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…
Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers). I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something rig…
Back when Resilio Sync was known as Bittorrent Sync, I proposed it as a file-sharing solution between multiple locations for the company I was working for. Immediately returned with an emphatic refusal to use any kind of software that had "Bittorrent" in the name, regardless of what it was for.
This is the same company where they picked up I had "Bittorrent Software" installed on my work laptop, and the people who ran the auditing software freaked out because, you know, it's Bittorrent! I told them it was a single icon in my PortableApps folder but they just kept saying "There's a Bittorrent client on your PC, you could be fired!" until I deleted the icon.
So yes, the fact it's associated with piracy has made a huge difference in some places.
You can send a 10TB hard drive faster by USPS than it would take to transmit across the internet.