Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

161–170 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#161

5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination 5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination 10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! https://www.tidbitsfortechs.com/2013/09/never-underestimate-... And since you now buy 1TB micro SD cards, so perhaps I'd split th…

10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! In Poland there is an option to send package via train for 8$. Just go to station and give package to conductor/train guard, few hours later receiver can get it at destination station. Sending HDDs full of raw video footage this way is very popular among video editors here.

This reminded me of the mail coaches in express trains of India. Some train routes still have a mail qualifier in their name. Mail by rail.

https://en.m.wikipedia.org/wiki/Express_trains_in_India

Re: Ask HN: What is your favorite method of sending large files?

#162

Earlier quoted context omitted.

Same thing happens in Greece with buses. It's a very popular way for moms to send home-made food to their single/university student sons.

In Croatia as well :)

Balkan brothers!

Re: Ask HN: What is your favorite method of sending large files?

#163
post #141

Earlier quoted context omitted.

Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers). I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something rig…

WebTorrent [1] can be used to run BitTorrent on the web. It's actually integrated into the Brave browser which is a nice feature. Other browsers might catch on soon. [1] https://webtorrent.io/

WebTorrent is pretty limited though:

> ... a browser-based WebTorrent client or "web peer" can only connect to other clients that support WebTorrent/WebRTC.

(from https://webtorrent.io/faq)

This essentially splinters the world into the bittorrent and webtorrent halves, which makes it far less useful.

Re: Ask HN: What is your favorite method of sending large files?

#164
post #134

5G from a kubernetes pod: netcat Send help... no seriously... how do I upload a file from a server I have a shell on but can't easily open ports to or install software on (and where the built in cp tool fails on big files...)?

One of the main uses I made https://patchbay.pub for was transferring files to servers/VMs where I don't have easy rsync access. You just need curl or another HTTP client installed.

Re: Ask HN: What is your favorite method of sending large files?

#165

5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination 5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination 10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! https://www.tidbitsfortechs.com/2013/09/never-underestimate-... And since you now buy 1TB micro SD cards, so perhaps I'd split th…

> Never underestimate the bandwidth of a station wagon full of backup tapes!

The exact quote [0] by Andrew Tanenbaum is: "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.".

Obligatory xkcd [1].

I met Andy Tanenbaum in 2008 at Vrije Universiteit in Amsterdam. I was the Technology Evangelist for AWS in Europe, giving a seminar there on cloud computing.

I loved him. Funny, smart. I didn't even know he was American until I met him in person (having studied computer science, you're not exactly sure of someone's nationality solely based on his last name).

[0]: https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum

[1]: https://what-if.xkcd.com/31/

Re: Ask HN: What is your favorite method of sending large files?

#166
post #2

Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

Magic Wormhole is good. These days I use croc,[0] which I find even better. [0]: https://github.com/schollz/croc

Oh this looks great! Thanks for this, checking it out.

Re: Ask HN: What is your favorite method of sending large files?

#167

Earlier quoted context omitted.

Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers). I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something rig…

> people need dedicated clients, instead of it working for example in browsers "Version 9.0 [...], released on June 20, 2006" "Opera introduced [...] a built-in BitTorrent client" [1] That's history now, of course, courtesy of today's browser monoculture. [1] https://en.wikipedia.org/wiki/History_of_the_Opera_web_brows...

Brave has a built-in torrent downloader.

Re: Ask HN: What is your favorite method of sending large files?

#168
post #2

Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

Magic Wormhole is good. These days I use croc,[0] which I find even better. [0]: https://github.com/schollz/croc

I never understood why croc chose to copy the features of Magic Wormhole but implement them in an incompatible way.

Partially out of my annoyance with this I implemented a port of Magic Wormhole in Go that fully interoperates with the official python version: https://github.com/psanford/wormhole-william

Re: Ask HN: What is your favorite method of sending large files?

#169

Netcat: $ nc -l 4242 > dest And then on the sending end: $ nc hostname 4242 This works great when you just need to get a file of any size from one machine to another and you’re both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.

and if you are not worried about corruption. You might at least want to run a checksum on both ends and compare them.

Netcat is tcp is "reliable" and has checksums for each packet, as long as the length is right you can be reasonably assured that the file transferred correctly.

Not that extra checksums are a bad idea...

Re: Ask HN: What is your favorite method of sending large files?

#170
post #2

Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

I once sent a huge file over airdrop and was shocked how much faster it was then sending through the local LAN.

some copy methods verify and check during the transfer operation, others do not, so the final speeds are very different between them
Post reply on HN