Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#61

Post by dane-pgp »

parliament32 wrote: post #27
cpach wrote: post #21

Earlier quoted context omitted.

GPG is just not a very good tool. I think 'tptacek explained it quite well in the article I linked. With that said, Magic Wormhole is also a very good tool for transferring files. It will encrypt in transit. So for many files, using a separate encryption tool is not necessary. (So far I haven’t tried it for large files.)

I want to comment on that article you keep referring to, but I don't want to clutter up the top of the thread so I'll do it here. The author really wants to dislike PGP, but the reason everyone trusts PGP is because it's been around forever. Yeah there've been deficiencies, just like there've been deficiencies in OpenSSL, but that doesn't make it a bad tool. I could go on but this xkcd sums it up: https://xkcd.com/23…

I wish the downvoters would explain what's wrong with the parent comment, as the counter-arguments raised by parliament32 seem completely reasonable to me. Perhaps it's just the length of the comment, and they would prefer it written up as a blog post somewhere.

Re: Ask HN: What is your favorite method of sending large files?

#62

Post by psanford »

ColanR wrote: post #35
cpach wrote: post #21

Earlier quoted context omitted.

GPG is just not a very good tool. I think 'tptacek explained it quite well in the article I linked. With that said, Magic Wormhole is also a very good tool for transferring files. It will encrypt in transit. So for many files, using a separate encryption tool is not necessary. (So far I haven’t tried it for large files.)

What no one seems to mention is that Magic Wormhole depends on the maintainer's server to negotiate transmission between clients. I don't like that requirement. Better to GPG encrypt and use bittorrent for a direct transfer. At least then we're using public trackers instead of some private server.

Its not a requirement that you use the default Wormole rendezvous or transit server. Its all open source you could run your own private servers if you wanted to (and its easy).

Of course the nice thing about Magic Wormhole is that its security does not depend on server components to be trustworthy. Use the default servers, use your own, use a different third party server, it doesn't matter, your data is still secure.

Edit: If you are worried about privacy magic-wormhole supports transit over tor.

Re: Ask HN: What is your favorite method of sending large files?

#64

Post by lowwave »

mullen wrote: post #45

If the data is below 50G in size and on my personal computer, then I just drop it into the Google Drive folder and it syncs to Google over night. When it is done, I export it and then send a link to the person(s). I pay $2 a month for the 100G account and I usually have about 50G disk space unused, so this is not an issue for me. If it is above 50G and on my personal computer, I encrypt the data and then physically m…

>If it is on a server (For example, a side project instance in AWS), then I drop it into S3 bucket, export it and send the URL to the recipient. I just eat the cost of them downloading it. Usually I am making money off the work anyways, so it is the cost of doing business.

Consider BunnyCDN way cheaper than s3.

Re: Ask HN: What is your favorite method of sending large files?

#66

Post by lowwave »

GRBurst wrote: post #46

It depends in whether I want to send it to someone in the local network or through the internet. For local network: I use miniserve ( https://github.com/svenstaro/miniserve ) which is just a simple http server. There are similar tools for when I want to share it from the smartphone. Through the internet it really varies: Sometimes it is Firefox send ( https://send.firefox.com/ ) For photos, I use a self hosted photo…

for local network. You can just do python3 -m http.server in the dir, or python -m SimpleHTTPServer (python2)

Or npm install -g http-server for nodejs,

then http-server

Local network is easy.

Re: Ask HN: What is your favorite method of sending large files?

#67

Post by k__ »

geocrasher wrote: post #57

5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination 5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination 10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! https://www.tidbitsfortechs.com/2013/09/never-underestimate-... And since you now buy 1TB micro SD cards, so perhaps I'd split th…

Just thought about AWS Snowmobile.

Re: Ask HN: What is your favorite method of sending large files?

#68

Post by banana_giraffe »

For things smaller than around 10gb, I tend to use S3. Either with pre-signed links behind a little bespoke portal that people can log into and see what files I've "shared" with them, or just direct S3 links for those that what to automate it all.

For bigger things, there are two basic paths:

If it's to someone that's not primary a tech person, then the data goes on a portable hard drive, and I either drive it to them, or mail it and walk them through accessing the data. In both cases I encrypt the data, generally with Veracrypt.

If it's to someone that won't mind using AWS's toolchain, and has decent connectivity, I'll use Snowcone or Snowball to get the data to S3 and give them S3 URLs.

I tend to get the data to S3 sooner or later so I can migrate it to Glacier. More than once I've had to recover GBs or TBs of data because the customer lost it, so I'm prepared now.

Re: Ask HN: What is your favorite method of sending large files?

#69

Post by theobeers »

kvn_95 wrote: post #2

Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

Magic Wormhole is good. These days I use croc,[0] which I find even better.

[0]: https://github.com/schollz/croc

Re: Ask HN: What is your favorite method of sending large files?

#70

Post by unilynx »

ashton314 wrote: post #55

Netcat: $ nc -l 4242 > dest And then on the sending end: $ nc hostname 4242 This works great when you just need to get a file of any size from one machine to another and you’re both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.

This is especially fun when combined with piping through tar, and adding pv in the mix for a transfer speed "progress" bar.

Fastest way to transfer a collection of files on a local network and doesn't require temporary storage for the archive

Post reply on HN