Ask HN: What is your favorite method of sending large files? • Hacker News Board

Re: Ask HN: What is your favorite method of sending large files?

#31

Post by emerged » Wed, Sep 02, 2020, 7:27 PM UTC

upload to s3 bucket, make public, send link

Re: Ask HN: What is your favorite method of sending large files?

#32

Post by _8j50 » Wed, Sep 02, 2020, 7:29 PM UTC

ngrok.io and similar reverse tunnels

Re: Ask HN: What is your favorite method of sending large files?

#33

Post by akerro » Wed, Sep 02, 2020, 7:29 PM UTC

>I'm also curious on how you'd send 10TB or more.

For my 12TiB of data I use Syncthing when I need to sync them more often, rsync.

I used rsync several times for billions of smaller files totalling to 300GiB, but really all depending on how I connect nodes. I prefer syncthing, but when only ssh is available, then rsync is good too.

Currently largest synced directory by syncthing (that shares usage stats) is over 61384 GiB :)) https://data.syncthing.net/

Re: Ask HN: What is your favorite method of sending large files?

#34

Post by celim307 » Wed, Sep 02, 2020, 7:30 PM UTC

auxym wrote: post #14

hotwire wrote: post #9

A station wagon full of tapes hurtling down the highway...

No one makes station wagons anymore, except luxury european makers :(

Subaru Outback’s

Re: Ask HN: What is your favorite method of sending large files?

#35

Post by ColanR » Wed, Sep 02, 2020, 7:31 PM UTC

cpach wrote: post #21

forgotmypw17 wrote: post #20

Earlier quoted context omitted.

That's said often about any tool which has been around long enough. People without experience come around and think they can replace an old tool with a better one, but it's usually only ignorance of either the complexity of the task, knowledge of using the tool properly, or both.

GPG is just not a very good tool. I think 'tptacek explained it quite well in the article I linked. With that said, Magic Wormhole is also a very good tool for transferring files. It will encrypt in transit. So for many files, using a separate encryption tool is not necessary. (So far I haven’t tried it for large files.)

What no one seems to mention is that Magic Wormhole depends on the maintainer's server to negotiate transmission between clients. I don't like that requirement. Better to GPG encrypt and use bittorrent for a direct transfer. At least then we're using public trackers instead of some private server.

Re: Ask HN: What is your favorite method of sending large files?

#36

Post by hprotagonist » Wed, Sep 02, 2020, 7:33 PM UTC

magic-wormhole: https://magic-wormhole.readthedocs.io/en/latest/

  pipx install magic-wormhole
  wormhole send /some/big/stupid/thing.tar.gz

Re: Ask HN: What is your favorite method of sending large files?

#37

Post by Apreche » Wed, Sep 02, 2020, 7:36 PM UTC

zepearl wrote: post #28

Personally I would use my own Nextcloud instance for up to 20-30GBs. Not sure about TBs. What about using "Firefox Send"? (I never used it so far) https://support.mozilla.org/en-US/kb/send-files-anyone-secur... I read that the limit is 1-2.5GBs => maybe you could break down the file and upload it in multiple pieces... . EDIT: oopps, Firefox Send doesn seem to be available anymore - https://support.mozilla.org/en-US/k…

People were using Firefox Send to "send" malware and other bad things to people. That's what happens if you let anyone anonymously host files on your domain for free.

Re: Ask HN: What is your favorite method of sending large files?

#38

Post by codethief » Wed, Sep 02, 2020, 7:38 PM UTC

Syncthing works pretty well in my experience, even across NAT. I recently shared 10GB worth of photos with a family member and ~30min later it was done.