Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your favorite method of sending large files?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 363 posts

Re: Ask HN: What is your favorite method of sending large files?

#141

Post by 3s »

libertine wrote: post #96
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers). I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something rig…

WebTorrent [1] can be used to run BitTorrent on the web. It's actually integrated into the Brave browser which is a nice feature. Other browsers might catch on soon.

[1] https://webtorrent.io/

Re: Ask HN: What is your favorite method of sending large files?

#142

Post by pessimizer »

mderazon wrote: post #92
livueta wrote: post #86

Bittorrent. No, really. Lots of nice behaviors when transferring large amounts of data between arbitrary endpoints. Transferring the torrent metadata is pretty trivial and can be done via a wide range of methods, and having that flexibility can be nice. Unlike HTTP, you get reasonable retry behavior on network hiccups. Also, more robust data integrity guarantees, though a manual hash test is probably a good idea eith…

Can you recommend easy software to do the creation? Don't you need a tracker as well?

With transmission-create you can create a torrent from the commandline. Load it into your bittorrent client, find its hash in the UI, and make it into a magnet link, like:

magnet:?xt=urn:btih:[this is where the hash goes]

Example:

magnet:?xt=urn:btih:a008663c4fc7e710bd59200e5ef99ba04f83f319

Re: Ask HN: What is your favorite method of sending large files?

#143

Post by nakodari »

I am running Jumpshare (https://jumpshare.com) and our users regularly send large files anywhere from 2GB to 100GB and in some cases even more. My recommendation is to use cloud providers for sharing files that are maximum 50GB in size. Please note that although you can share even bigger files you have to consider the reliability of the internet provider of both the sender and receiver. For files bigger than 50GB in size, I would recommend a P2P solution.

Re: Ask HN: What is your favorite method of sending large files?

#145

Post by jpxw »

terramex wrote: post #131
geocrasher wrote: post #57

5GB on my local network: Windows File Share, Rsync, or HTTP depending on source/destination 5GB on Internet: Upload to my OVH VPS and HTTP or Rsync it to its destination 10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! https://www.tidbitsfortechs.com/2013/09/never-underestimate-... And since you now buy 1TB micro SD cards, so perhaps I'd split th…

10TB, local or Internet: Physical Hard Drive. Never underestimate the bandwidth of a station wagon full of backup tapes! In Poland there is an option to send package via train for 8$. Just go to station and give package to conductor/train guard, few hours later receiver can get it at destination station. Sending HDDs full of raw video footage this way is very popular among video editors here.

That’s a really cool idea actually

Re: Ask HN: What is your favorite method of sending large files?

#147

Post by anderspitman »

I've been working on solutions in this space for a couple years now. IMO making data on your local device available to others via HTTP range requests is the sweet spot between storing your data in the cloud and going full p2p.

Here's a couple of my projects:

https://patchbay.pub/

code[0]

Sender:

  curl https://patchbay.pub/random-channel/filename.bin --data-binary @filename.bin
Receiver:

  curl -O https://patchbay.pub/random-channel/filename.bin
https://fbrg.xyz

code[1]

This one works in the browser. You select a file and it gives you a link you can share with others. The underlying tech[2] is more complicated (relies on WebSockets) than patchbay, but the server is currently more reliable. I'm in the process of improving the patchbay server to be more robust, and I think it will eventually replace fibridge completely.

My current project is building something along the lines of Google Drive, but much simpler and designed to be self-hosted from within your home.

[0]: https://github.com/patchbay-pub/

[1]: https://github.com/anderspitman/fibridge-proxy-rs

[2]: https://github.com/omnistreams/omnistreams-spec

Re: Ask HN: What is your favorite method of sending large files?

#148

Post by WarOnPrivacy »

I and my customers all have symmetric FTTH. When I'm copying a file from me to me (working remotely), I use a VPN. These are usually .vhdx files, ranging from 6GB to 300GB. I copy them to here for troubleshooting, then I send them back.

When I want to make something large available to someone else, I post it on my local webserver. It points to an NAS on my LAN; anyone in the house can just drop whatever on it and hand out a link.

Re: Ask HN: What is your favorite method of sending large files?

#149

Post by onelastjob »

For transfers on the smaller side, I'd have a look at https://massive.io/ I've done a fair bit of looking into options for sending files online and I like their pricing model a lot because it is purely usage based and has no caps. It's $0.25/GB for downloads.

For larger transfers, I'd look at File Catalyst Spaces. If you have a bigger budget, you could look at IBM Aspera or Media Shuttle.

Post reply on HN