Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

Post reply on HN
Print view View original

221–230 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#221

Post by orionblastar »

When I worked as a federal contractor in 1996-1997 we used password protected FTP servers to store giant text files for import to a database. Later on they moved it to a website you login and download the text file. Since it was so long it took almost all day and if it aborted had to start all over again.

I had to keep Netscape open because it showed the status of the file download. People asked why my web browser was open all day, it was for the giant download that is part of my job.

Re: Ask HN: What is your favorite method of sending large files?

#222

Post by qrv3w »

spurgu wrote: post #186
theobeers wrote: post #69

Earlier quoted context omitted.

Magic Wormhole is good. These days I use croc,[0] which I find even better. [0]: https://github.com/schollz/croc

I don't see an option to send text (browsed through README and Issues), is this not possible with croc? With Magic Wormhole it's the default to get a message prompt when running "wormhole send". Or you can do it as a oneliner with "wormhole send --text 'something'". Wormhole William can do this as well IIRC. I use MWH often for sending stuff like URL's, passwords, API keys... having to create a file for that is quite…

Just added it now. Send with croc send --text "hello, world" [1]

[1]: https://github.com/schollz/croc/releases/tag/v8.2.0

Re: Ask HN: What is your favorite method of sending large files?

#223

Post by yjftsjthsd-h »

hoorayimhelping wrote: post #205
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

> Just encrypt your stuff beforehand. This is such a facepalm moment. "This service is awesome, it does everything you need, it's reliable, it's safe, it's practical, and all you have to do is encrypt your files, share encryption keys, validate identities, and establish trust with the person you're sharing with. Also if you screw it up in any way, your file is public forever. It's so rad!"

No, "here's a magnet link, and here's the matching 50-char password; don't share the password with anyone you don't want have file". Just because you can make security more complicated doesn't mean you're obligated to.

Re: Ask HN: What is your favorite method of sending large files?

#224

Post by ekianjo »

hoorayimhelping wrote: post #205
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

> Just encrypt your stuff beforehand. This is such a facepalm moment. "This service is awesome, it does everything you need, it's reliable, it's safe, it's practical, and all you have to do is encrypt your files, share encryption keys, validate identities, and establish trust with the person you're sharing with. Also if you screw it up in any way, your file is public forever. It's so rad!"

> Also if you screw it up in any way, your file is public forever.

You can literally say that for ANYTHING out there. Not sure what point you are trying to make. Let me guess, "Encryption is hard?". Well there's no free lunch.

Re: Ask HN: What is your favorite method of sending large files?

#225

Post by clankyclanker »

qrv3w wrote: post #180
psanford wrote: post #168

Earlier quoted context omitted.

I never understood why croc chose to copy the features of Magic Wormhole but implement them in an incompatible way. Partially out of my annoyance with this I implemented a port of Magic Wormhole in Go that fully interoperates with the official python version: https://github.com/psanford/wormhole-william

I initiated the croc project. Others have asked me the same thing, so I'll bite: I chose not to copy the features of magic wormhole because it seemed to me (at the time) to be very complicated and lacking in features crucial to myself (namely restarting transfers [1], sending folders without zipping them). This has been to my benefit, I think, because I can iterate on croc quickly without having to conform to the wor…

Would it be possible to sniff the protocol on the wire and provide some sort of cross-compatible magic wormcroc service?

Re: Ask HN: What is your favorite method of sending large files?

#226

Post by sillysaurusx »

Syncthing. https://syncthing.net/

It's like a private dropbox.

For files on the order of https://techcrunch.com/2017/06/27/magic-wormhole-is-a-clever...

`sudo pip3 install magic-wormhole` is an automatic "must run this command" on all my servers. Simple, secure way of sending files from A to B, with no bullshit. No account creation, even.

Re: Ask HN: What is your favorite method of sending large files?

#227

Post by aladine »

mkl wrote: post #218
cprecioso wrote: post #113

Earlier quoted context omitted.

Firefox Send doesn’t work anymore

That's unfortunate. That was my go-to method. From https://send.firefox.com/ : "Firefox Send is temporarily unavailable while we work on product improvements. We appreciate your patience while we make the Firefox Send experience better."

I was said a bit because it is a nice product developed by Mozilla.

Re: Ask HN: What is your favorite method of sending large files?

#228

Post by KindOne »

temp007007 wrote: post #217
akerro wrote: post #33

>I'm also curious on how you'd send 10TB or more. For my 12TiB of data I use Syncthing when I need to sync them more often, rsync. I used rsync several times for billions of smaller files totalling to 300GiB, but really all depending on how I connect nodes. I prefer syncthing, but when only ssh is available, then rsync is good too. Currently largest synced directory by syncthing (that shares usage stats) is over 6138…

Is there an iOS client for Syncthing?

No.

https://docs.syncthing.net/users/faq.html#why-is-there-no-io...

Re: Ask HN: What is your favorite method of sending large files?

#229

Post by asveikau »

arsome wrote: post #178
myself248 wrote: post #121

Earlier quoted context omitted.

> Unlike HTTP, you get reasonable retry behavior on network hiccups Seriously. Zmodem/resume was doing better in 1986.

wget -c, it ain't hard.

You need to notice the connection died and do that manually, or think ahead and have your script do it in a loop. A bt client already has the reconnect logic. And it will hash the contents to make sure it worked correctly.

Re: Ask HN: What is your favorite method of sending large files?

#230

Post by [deleted] »

livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

[deleted]
Post reply on HN