Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

321–330 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#323

Netcat: $ nc -l 4242 > dest And then on the sending end: $ nc hostname 4242 This works great when you just need to get a file of any size from one machine to another and you’re both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.

[deleted]

Re: Ask HN: What is your favorite method of sending large files?

#324
For files less than 1GB I tend to use Telegram. It obviously has the downside of uploading to server X and then downloading from server X, but usually if I am sending a file to one person, I'll likely be sending it to others too, and therefore the ability to forward the file to others arbitrarily after the fact actually proves to be a pretty useful quality. If I care about the datas security, an encrypted 7z container or something will do.

For files I need more control over that are less than say 5gb, I tend to scp them to a web server I control, so that I can delete them afterwards.

For files larger than that, I'll use a private Bittorrent file. It's very rare I need to transfer files this large, but I really like this solution.

Re: Ask HN: What is your favorite method of sending large files?

#325

In the enterprise space, there is dedicated software for this function. It uses an UDP protocol designed directly for efficiency (less overhead, vs. TCP), and builds in resilience such that it'll work over connections with lots of packet drops, or connections with high latency or low reliability. https://gojetstream.io/

Interesting. Is this implemented similiar to QUIC as some kind of connection oriented protocol on top of UDP?

Re: Ask HN: What is your favorite method of sending large files?

#326

Earlier quoted context omitted.

I want to comment on that article you keep referring to, but I don't want to clutter up the top of the thread so I'll do it here. The author really wants to dislike PGP, but the reason everyone trusts PGP is because it's been around forever. Yeah there've been deficiencies, just like there've been deficiencies in OpenSSL, but that doesn't make it a bad tool. I could go on but this xkcd sums it up: https://xkcd.com/23…

I wish the downvoters would explain what's wrong with the parent comment, as the counter-arguments raised by parliament32 seem completely reasonable to me. Perhaps it's just the length of the comment, and they would prefer it written up as a blog post somewhere.

> I wish the downvoters would explain what's wrong with the parent comment,

One obvious guess is contradicting Latacora. 'tptacek is well known here, his name alone give significant weight to anything he writes.

In any case, they have valid points. PGP was written at a time we didn't understand cryptography as well as we do now. We can do better. Have done better, if half of what I've heard about Age is true.

Absurd complexity: We can definitely do simpler than PGP, at no loss of functionality.

Swiss Army Knife design: I think I disagree with Latacora there. Doing many things doesn't mean you have to do them poorly. There's no material difference between having 3 programs, and having one programs with 3 options, at least on the command line. If PGP does anything poorly, it's for other reasons.

Mired In Backwards Compatibility: well that depends. It makes sense that PGP can decrypt old obsolete ciphers & formats. Ability to encrypt to those same old stuff wouldn't. For instance, PGP longer be able to generate RSA keys at all. Then, one year later, once all RSA keys have expired, new PGP versions should no longer be able to encrypt to RSA keys at all. (In an ideal world. More realistically, we should wait a couple more years.) Only the ability to decrypt old messages should be kept until the end of times.

Obnoxious UX: I don't know enough to have an opinion.

Long term secrets: Sure they're bad, but I don't think we can avoid them. People need your public key to send you anything, so it can't be too short lived. My guess here is that Latacora is attacking the whole file encryption + web of trust thing, not PGP in particular.

Broken Authentication: If attackers can trick PGP decoders into decrypting forged messages, that's fairly critical, and should be fixed even if it breaks backwards compatibility (we could have an optional `-legacy` flag or something to compensate). Now if you go sign and encrypt… well there are two possibilities: if you sign then encrypt, you run into the cryptographic doom principle: the decoder will decrypt then verify, which creates the temptation to process unauthenticated data. Many vulnerabilities have been caused by such errors. If you encrypt then sign, you reveal to the entire world that you signed this particular ciphertext. Not the kind of data most people would like to leak. In my opinion what we really want is authenticated key exchange followed by AEAD. With the Noise X pattern, you'd even hide your identity from snoopers.

Incoherent Identity: Okay, they're clearly attacking the very notion of web of trust, not PGP specifically. They say it doesn't work, but I'd like to know why. First, I'm not sure I want to take their word for it, and second, the causes might be fixable.

Leaks Metadata: that one is clearly avoidable. Noise X for instance uses an ephemeral key to encrypt the transmitted public keys, and the recipient's key is implicit. Can't know who the message is for (nor from) without the recipient's private key.

No Forward Secrecy: Different use case indeed. Again, Latacora is attacking the very notion of file encryption, not PGP specifically.

Clumsy Keys: I'm with Latacora on this one. The 50 lines SSH keys are clearly RSA based, and as such obsolete. Modern keys use elliptic curves, and those take one line, which is more easily copy & pasted in various contexts. Arguably a detail, though.

Negotiation: It's more than just backwards compatibility. Backwards compatibility can be achieved with a simple version number. If instead we have a range of algorithms to chose from, things get more complicated. Now, you can't avoid the need for different kinds of encryption: you can send against a public key, or you can encrypt with a password. Possibly both. Beyond that however it's simpler to have a version number that hints at a single public key encryption and a single password based encryption.

Janky Code: Can't judge for myself. I can guess however that much of it is caused by the (ever evolving) PGP specifications themselves. Probably more a consequence of all the other issues than a separate problem. Still, I think we can do much better. I mean, I've written an entire crypto library, and people give me hell for a single vulnerability in over 3 years. 27 CVE in comparison would be worth burning in Crypto Hell for a long time.

Re: Ask HN: What is your favorite method of sending large files?

#327
post #222
post #186

Earlier quoted context omitted.

I don't see an option to send text (browsed through README and Issues), is this not possible with croc? With Magic Wormhole it's the default to get a message prompt when running "wormhole send". Or you can do it as a oneliner with "wormhole send --text 'something'". Wormhole William can do this as well IIRC. I use MWH often for sending stuff like URL's, passwords, API keys... having to create a file for that is quite…

Just added it now. Send with croc send --text "hello, world" [1] [1]: https://github.com/schollz/croc/releases/tag/v8.2.0

Haha wow, thanks! I'll give it a spin today!

Re: Ask HN: What is your favorite method of sending large files?

#329
post #180

Earlier quoted context omitted.

I initiated the croc project. Others have asked me the same thing, so I'll bite: I chose not to copy the features of magic wormhole because it seemed to me (at the time) to be very complicated and lacking in features crucial to myself (namely restarting transfers [1], sending folders without zipping them). This has been to my benefit, I think, because I can iterate on croc quickly without having to conform to the wor…

Thank you. Croc is fantastic because it solves a problem that no other tool does. Magic Wormhole stumbles at the first hurdle (installability).

I'm sorry but I don't understand the hurdle?

As I mentioned in another thread installing Magic Wormhole is _easy_, just "brew install magic-wormhole" or "sudo apt-get install magic-wormhole" (or distro equivalent). Windows might be tricky but don't see how croc does it much better. I hadn't heard of Scoop but seems like you'd need to install _that_ first (which has its own dependencies). That said I haven't used Windows in many many years, maybe Scoop is the standard nowadays like Homebrew is for Mac? In which case I'd give a point to croc.

Also, with a Go app you could just share/distribute the binary, but then you also need to know where to put it. And you could do that with Magic Wormhole as well.

Or a step better like using something like PyInstaller (wonder why author hasn't done that?)..

Re: Ask HN: What is your favorite method of sending large files?

#330

For anything 5GB plus - external drive & cycle over The only person I'm sharing with lives near me, so sneakernet is the most convenient. I once hit a transfer speed of 30TBs/hour carrying a box of hard drives home from work.

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."

– Andrew Tanenbaum, 1981

Post reply on HN