Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)
Ask HN: What is your favorite method of sending large files?
331–340 of 363 posts
Re: Ask HN: What is your favorite method of sending large files?
#332Earlier quoted context omitted.
BitTorrent is the best for sending large files. I wish there existed a tool to make torrent creation easier, and I also wish my computer could seed. Every time I create a torrent, nobody can discover my server, even though the port is open and even if I add the server IP as a peer directly onto the client. I have no trouble seeding in swarms, only when I create torrents. Other than that, BT is ideal.
Maybe you're only able to make outbound connections and not able to accept incoming connections? Try port scanning your PC from the internet
Re: Ask HN: What is your favorite method of sending large files?
#333Earlier quoted context omitted.
I initiated the croc project. Others have asked me the same thing, so I'll bite: I chose not to copy the features of magic wormhole because it seemed to me (at the time) to be very complicated and lacking in features crucial to myself (namely restarting transfers [1], sending folders without zipping them). This has been to my benefit, I think, because I can iterate on croc quickly without having to conform to the wor…
Can you write what is that hadcoded relay in croc? Who maintains it, how does it handle the increase in demand or outages etc?
In any case, I encourage everyone to setup their own relay. Its easy. Make sure ports 9009-9014 are open and then run `croc relay`.
Connecting to your relay is as easy as `croc --relay ADDRESS:9009 send X`. And if you add the `--remember` flag you only have to tell it which relay to use once.
Re: Ask HN: What is your favorite method of sending large files?
#334Earlier quoted context omitted.
I initiated the croc project. Others have asked me the same thing, so I'll bite: I chose not to copy the features of magic wormhole because it seemed to me (at the time) to be very complicated and lacking in features crucial to myself (namely restarting transfers [1], sending folders without zipping them). This has been to my benefit, I think, because I can iterate on croc quickly without having to conform to the wor…
> so I'll bite Just curious: pun intended?
Re: Ask HN: What is your favorite method of sending large files?
#335Earlier quoted context omitted.
I initiated the croc project. Others have asked me the same thing, so I'll bite: I chose not to copy the features of magic wormhole because it seemed to me (at the time) to be very complicated and lacking in features crucial to myself (namely restarting transfers [1], sending folders without zipping them). This has been to my benefit, I think, because I can iterate on croc quickly without having to conform to the wor…
Would it be possible to sniff the protocol on the wire and provide some sort of cross-compatible magic wormcroc service?
Re: Ask HN: What is your favorite method of sending large files?
#336Earlier quoted context omitted.
Thank you. Croc is fantastic because it solves a problem that no other tool does. Magic Wormhole stumbles at the first hurdle (installability).
I'm sorry but I don't understand the hurdle? As I mentioned in another thread installing Magic Wormhole is _easy_, just "brew install magic-wormhole" or "sudo apt-get install magic-wormhole" (or distro equivalent). Windows might be tricky but don't see how croc does it much better. I hadn't heard of Scoop but seems like you'd need to install _that_ first (which has its own dependencies). That said I haven't used Wind…
I agree installation for magic wormhole is easy for Mac/Linux, but I think its not easy on Windows. Windows is very common (77% of the market share for desktops [1]) so I wanted to keep croc easy to install for Windows.
In fact, if you are just receiving a file on Windows, all you have to do is download the binary from releases [2], unzip it, and double click on the binary - no terminal experience required. (Sending a file does require using a terminal, but all CLI apps have that hurdle...).
[1]: https://gs.statcounter.com/os-market-share/desktop/worldwide
Re: Ask HN: What is your favorite method of sending large files?
#337Netcat: $ nc -l 4242 > dest And then on the sending end: $ nc hostname 4242 This works great when you just need to get a file of any size from one machine to another and you’re both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.
Re: Ask HN: What is your favorite method of sending large files?
#338Re: Ask HN: What is your favorite method of sending large files?
#339Earlier quoted context omitted.
Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly. For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.
For some reason I haven’t understood, syncthing would often miss transferring files but mark the entire transfer/sync as complete. No errors in the logs either. I tried it for a few months earlier this year, then gave up and switched to plain rsync in a shell. Based on that sample of one personal experience, I wouldn’t recommend syncthing to anyone. Note: I was using syncthing for a one way sync all the time, with se…
This sounds like a pretty severe bug, so if you still have the systems you were able to produce the behavior on it might be helpful to reproduce it and submit an issue.
Re: Ask HN: What is your favorite method of sending large files?
#340Earlier quoted context omitted.
This is especially fun when combined with piping through tar, and adding pv in the mix for a transfer speed "progress" bar. Fastest way to transfer a collection of files on a local network and doesn't require temporary storage for the archive
What's an example of the syntax of this? I remember seeing a graybeard do something like this to huck some files to another machine.
tar cf - files morefiles | ssh user@dest "tar xf -"
If you want to insert, say, compression: tar cf - files morefiles | zstd | ssh user@dest "zstd -d | tar xf -"