Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

Post reply on HN
Print view View original

241–250 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#241

Post by paxys »

libertine wrote: post #96
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers). I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something rig…

BitTorrent didn't catch on simply because there aren't very many actual use cases for it. The majority of P2P content distribution will always be illegal (due to the nature of copyright), and so the client-server model is much more representative of the real world. It is technologically pretty neat though, and some publishers do actually use the protocol for things like software updates.

Re: Ask HN: What is your favorite method of sending large files?

#242

Post by asdff »

jpxw wrote: post #145
terramex wrote: post #131

Earlier quoted context omitted.

10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! In Poland there is an option to send package via train for 8$. Just go to station and give package to conductor/train guard, few hours later receiver can get it at destination station. Sending HDDs full of raw video footage this way is very popular among video editors here.

That’s a really cool idea actually

Amtrak does it too

Re: Ask HN: What is your favorite method of sending large files?

#243

Post by farmdawgnation »

StavrosK wrote: post #185
lowwave wrote: post #64

Earlier quoted context omitted.

>If it is on a server (For example, a side project instance in AWS), then I drop it into S3 bucket, export it and send the URL to the recipient. I just eat the cost of them downloading it. Usually I am making money off the work anyways, so it is the cost of doing business. Consider BunnyCDN way cheaper than s3.

BunnyCDN is a CloudFront alternative, not an S3 alternative.

They also offer an object storage product, it seems: https://bunnycdn.com/solutions/cdn-cloud-storage

Re: Ask HN: What is your favorite method of sending large files?

#244

Post by Seb-C »

I have been using mega.nz to synchronize and backup my files for years, and so far I am very happy with it.

Everything works as expected, there is a lot of storage space and end-to-end encryption. There are clients for all platforms and it just works.

I can easily create secure share links whenever I want.

The only downsides are that the browser-based app is very slow to start once you have lots of files, and the android client does not have the same synchronization capabilities.

Re: Ask HN: What is your favorite method of sending large files?

#246

Post by flyingfences »

pxx wrote: post #213
ytjohn wrote: post #200

Earlier quoted context omitted.

In the US, you can ship using Greyhound bus lines. I wouldn't recommend it for this scenario, but I know people that ship auto parts around this way. Ship up to 100lbs for some incredibly cheap rates. http://www.shipgreyhound.com/

This looks incredibly expensive and not competitive at all. Over $20 for a 1 lb package, over $60 for a 50 lb package from Oakland to Los Angeles and a quoted time of four days station to station. I can overnight a 50 lb package for $35, door to door, over that distance with the appropriate FedEx account and discounts. The ground service would cost me $30 without any discounts...

In my (limited) experience, the Greyhound shipping service becomes competitive when shipments are very bulky (like body parts) more so than when they are heavy.

Re: Ask HN: What is your favorite method of sending large files?

#247

Post by spieglt »

mettamage wrote: post #4
kvn_95 wrote: post #2

Magic Wormhole ( https://magic-wormhole.readthedocs.io/en/latest/ ) if it's between OSes. Between OSX, AirDrop works very well. I have sent >10GB files between Macs, quite quick as well. I have never send a 10TB file so I wouldn't know. None of my drives are that large yet :)

I use the magic of AirDrop as well :) Unfortunately, sometimes I'm on my Linux machine (in which case python3 -m http.server works just fine), or need to send a file to someone else and then I'd like to have at least some form of monitoring/progress. Edit: magic wormhole is awesome! Thanks! :D

I wrote github.com/spieglt/flyingcarpet because I needed AirDrop on my Linux and Windows machines, pretty much does the same thing as running a Python server but over ad hoc wifi.

Re: Ask HN: What is your favorite method of sending large files?

#248

Post by egypturnash »

Apple Mail has a 30-day storage zone for large email attachments that it automatically offers when you attach big stuff, and I usually use that.

When I don't I usually just zip it and upload it to the media section of my Wordpress-managed website.

I've never generated files in the terabyte range, so I don't worry about that.

Re: Ask HN: What is your favorite method of sending large files?

#249

Post by jivank »

arsome wrote: post #179
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly. For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.

I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary.

The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discovery, which would be okay if you use a tracker behind the LAN (though the torrent would need to be modified if the hostname/IP changes). Since you can't configure other people's clients I found it simpler to use Aria2 that is preconfigured to disable PEX and DHT.

Combine this with Metalink or Web Seeds, so that you can have a initial seeder based on HTTP. I think using IPFS would be a great web seed as long as the their gateway and Cloudfare's continues to stay up. IPFS creates a permanent URL if you will, so no need to worry about dynamic IPs or domains. It would be great if IPFS would have a smaller binary (right now it is around 20MB compressed) and had a way to get a file and "seed" it like a torrent. But for now I think torrent is mature enough.

Re: Ask HN: What is your favorite method of sending large files?

#250

Post by 1ark »

10TB or more, not sure. Anecdotally I did have to send GBs of home directory locally recently. Fastest way I found out the hard way was mounting SMBFS and `tar` it there. But of course it is unreliable without resume etc.

For the thing you did, these could have worked.

https://file.pizza

https://instant.io

https://www.sharedrop.io

Post reply on HN