Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

Post reply on HN
Print view View original

251–260 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#251

Post by kortex »

unilynx wrote: post #70
ashton314 wrote: post #55

Netcat: $ nc -l 4242 > dest And then on the sending end: $ nc hostname 4242 This works great when you just need to get a file of any size from one machine to another and you’re both on the same network. Are used this a lot at one of my offices to schlep the files around between a few of the machines we had.

This is especially fun when combined with piping through tar, and adding pv in the mix for a transfer speed "progress" bar. Fastest way to transfer a collection of files on a local network and doesn't require temporary storage for the archive

What's an example of the syntax of this? I remember seeing a graybeard do something like this to huck some files to another machine.

Re: Ask HN: What is your favorite method of sending large files?

#252

Post by ip26 »

pxx wrote: post #213
ytjohn wrote: post #200

Earlier quoted context omitted.

In the US, you can ship using Greyhound bus lines. I wouldn't recommend it for this scenario, but I know people that ship auto parts around this way. Ship up to 100lbs for some incredibly cheap rates. http://www.shipgreyhound.com/

This looks incredibly expensive and not competitive at all. Over $20 for a 1 lb package, over $60 for a 50 lb package from Oakland to Los Angeles and a quoted time of four days station to station. I can overnight a 50 lb package for $35, door to door, over that distance with the appropriate FedEx account and discounts. The ground service would cost me $30 without any discounts...

It may be competing with freight, instead of fedex. Auto parts don't always fit in a Priority (Small) box.

Re: Ask HN: What is your favorite method of sending large files?

#253

Post by kortex »

Matthias247 wrote: post #231
gpm wrote: post #169

Earlier quoted context omitted.

Netcat is tcp is "reliable" and has checksums for each packet, as long as the length is right you can be reasonably assured that the file transferred correctly. Not that extra checksums are a bad idea...

TCP indeed has checksums! But they are known not to offer ultimate protection. I had to deal with hosts corrupting TCP streams due to bad hardware before. Which was luckily detected by higher level checksums.

Is there a streamlike tool (a la sed, gzip, etc) that applies forward error correction? I've been looking for such a thing but can't find one that scratches the itch.

Re: Ask HN: What is your favorite method of sending large files?

#254

Post by motohagiography »

Bike courier with checkered past, simultaneously on the run from organized crime clans and mercenaries working for retired spies, probably has mirror shade implants, lives in squat, punk haircut, etc.

Didn't realize there was another way.

Re: Ask HN: What is your favorite method of sending large files?

#255

Post by brink »

jivank wrote: post #249
arsome wrote: post #179

Earlier quoted context omitted.

Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly. For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.

I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary. The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discover…

Why not use Syncthing or Resilio Sync for LAN parties?

Re: Ask HN: What is your favorite method of sending large files?

#256

Post by jivank »

brink wrote: post #255
jivank wrote: post #249

Earlier quoted context omitted.

I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary. The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discover…

Why not use Syncthing or Resilio Sync for LAN parties?

I have considered it. The one thing about Syncthing is that there is no true readonly way to sync. For example, someone may accidentally extract an archive in the Syncthing folder and it will sync everywhere. If Syncthing gets this feature I think I would be 100% onboard with it for LAN parties.

Resilio I believe supports it but I would prefer an opensource alternative.

https://github.com/syncthing/syncthing/issues/62

Re: Ask HN: What is your favorite method of sending large files?

#257

Post by tclancy »

flyingfences wrote: post #246
pxx wrote: post #213

Earlier quoted context omitted.

This looks incredibly expensive and not competitive at all. Over $20 for a 1 lb package, over $60 for a 50 lb package from Oakland to Los Angeles and a quoted time of four days station to station. I can overnight a 50 lb package for $35, door to door, over that distance with the appropriate FedEx account and discounts. The ground service would cost me $30 without any discounts...

In my (limited) experience, the Greyhound shipping service becomes competitive when shipments are very bulky (like body parts) more so than when they are heavy.

You mean auto body parts, right?

Re: Ask HN: What is your favorite method of sending large files?

#258

Post by natch »

There was some tool for doing this securely on the command line. You run it with the file name and it gives you back a token. You give the token to your friend, they run the same tool with the token, and it downloads the file. Can’t for the life of me find it or remember what it was called. Wormhole?

Edit: I guess it was magic wormhole, discussed elsewhere in this thread.

Re: Ask HN: What is your favorite method of sending large files?

#259

Post by ta95846893 »

parliament32 wrote: post #104
bscphil wrote: post #80

Earlier quoted context omitted.

> By default, GPG keys expire in a year. This speaks to the article's complaint that GPG is usually the wrong tool for the job. For example if you just need to transfer a file securely (and have a fast, reliable internet connection on both ends and don't need to worry about active tracking of metadata), you can use Magic Wormhole (or a similar PAKE system) to do it. Imagine two scenarios: one with GPG and one with PA…

>With PAKE the keys used to exchange the data are ephemeral, and so this isn't even a possibility. AFAIK this isn't really true. If the adversary captures the initial key exchange plus all the data (ie the full transaction), then later discovers your PSK, they'll be able to decrypt. The only case where this helps you is if they capture some packets out of the middle without the initial handshake. >authenticated encry…

> AFAIK this isn't really true. If the adversary captures the initial key exchange plus all the data (ie the full transaction), then later discovers your PSK, they'll be able to decrypt. The only case where this helps you is if they capture some packets out of the middle without the initial handshake.

Someone can correct me if I'm wrong, but I believe the idea behind a PAKE is that the password only authenticates the key exchange and doesn't contribute to it. So if you record all transmitted data you still need to break the key exchange which should have used a bunch of random bytes from both parties that are thrown away after use. The password is only there to prevent MITM, not to derive keys.

I believe magic wormhole uses SPAKE2, which has perfect forward secrecy. When using passwords to secure transmitted files it's really important to have forward secrecy otherwise you risk the transmission being recorded and the password being attacked offline which depending on your password strength might lead to trivially decrypting the data.

Re: Ask HN: What is your favorite method of sending large files?

#260

Post by ta95846893 »

parliament32 wrote: post #17
cpach wrote: post #11

Related: If you need to transfer sensitive data over Bittorent, Age is a good tool for encrypting it before transmission. https://github.com/FiloSottile/age

Why a new tool? How is this better than gpg symmetric encryption, considering gpg is installed/available effectively everywhere? Encrypt: gpg --symmetric file.dat (enter a password) Decrypt: gpg --decrypt file.dat.gpg > file.dat (enter the password)

gpg symmetric encryption just derives a key from the password so theres no forward secrecy. Using bittorrent in particular opens you up to the risk of third parties getting ahold of your encrypted file and without forward secrecy the can attack the password offline which is typically much smaller that 256 bits.

Using Magic Wormhole would seem to be far more secure than gpg and bittorrent since it has less risk of third party interception and uses SPAKE2 which has perfect forward secrecy.

If I had to use bittorrent to transfer a sensitive file I would generating a random key and encrypt, then share the key over a different channel.

Post reply on HN