Datapacket has a number of dedicated server configurations in various locations and offers unmetered connections:
Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
131–140 of 199 posts
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#132Cloudflare R2 or Oracle Cloud Infra or Hetzner
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#133If price is a consideration, you might consider two 10 TB hard drives on machines on two home gbps Internet connections. It's highly unlikely that both would go down at the same time, unless they were in the same area, on the same ISP.
How do you set up load balancing for those two connections? That is yourdomain.com -> IP_ISP1, IP_ISP2 Going the other way from yourserver -> outside would indicate some sort of bonding setup. It is not trivial for a home lab. I use 3 ISPs at home and just keep each network separate (different hardware on each) even though in theory the redundancy would be nice.
Just use two A records for the one DNS name, and let the clients choose.
The other way is to have two names, like dl1 and dl2, and have your download web page offer alternating links, depending on how the downloads are handled.
You very rarely can do multi-ISP bonding, often not even with multiple lines from the same ISP, unfortunately.
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#134I'd suggest looking into "seedboxes" which are intended for torrenting. I suspect the storage will be a bigger concern. Seedhost.eu has dedicated boxes with 8TB storage and 100TB bandwidth for €30/month. Perhaps you could have that and a lower spec one to make up the space. Prices are negotiable so you can always see if they can meet your needs for cheaper than two separate boxes.
> I'd suggest looking into "seedboxes" which are intended for torrenting.
Though be aware that many (most?) seedbox arrangements have no redundancy, in fact some are running off RAID0 arrays or similar. Host has a problem like a dead drive: bang goes your data. Some are very open about this, afterall for the main use case cheap space is worth the risk, some far less so…
Of course if the data is well backed up elsewhere or otherwise easy to reproduce or reobtain this may not be a massive issue and you've just got restore time to worry about (unless one of your backups can be quickly made primary so restore time is as little as a bit of DNS & other configuration work).
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#135you can definitely do this at home on the cheap. As long as you have a decent internet connection, that is ;) 10TB+ harddisks are not expensive, you can put them in an old enclosure together with a small industrial or NUC PC in your basement
I current have 45 WUH721414ALE6L4 drives in a Supermicro JBOD SC847E26 (SAS2 is way cheaper than SAS3) connected to an LSI 9206-16e controller (HCL reasons) via hybrid Mini SAS2 to Mini SAS3 cables. The SAS expanders in the JBOD are also LSI and qualified for the card. The hard drives are also qualified for the SAS expanders. I tried this using Pine ROCKPro64 to possibly install Ceph across 2-5 RAID1 NAS enclosures.…
What's your redundancy setup in the 45 drive configuration? I would guess 20-22 mirrors with 1-5 hotspares, but it's not clear.
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#136You could do this for about $1k/mo with Linode and Wasabi. For FastComments we store assets in Wasabi and have services in Linode that act as an in-memory+on disk LRU cache. We have terabytes of data but only pay $6/mo for Wasabi, because the cache hit ratio is high and Wasabi doesn't charge for egress until your egress is more than your storage or something like that. The rest of the cost is egress on Linode. The ni…
EDIT it would be ~$500 through linode + wasabi. I was thinking of included bandwidth, but if you just pay overages it's cheaper.
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#137Depends on what exactly you want to do with it. Hetzner has very cheap Storage boxes (10TB for $20/month with unlimited traffic) but those are closer to FTP boxes with a 10 connection limit. They are also down semi-regularly for maintenance. For rock-solid public hosting Cloudflare is probably a much better bet, but you're also paying 7 times the price. More than a dedicated server to host the files, but you get more…
> Hetzner has very cheap Storage boxes (10TB for $20/month with unlimited traffic) * based on fair use at 250 TB/mo: > In order to continue hosting your servers with us, the traffic use will need to be drastically reduced. Please check your servers and confirm what is using so much traffic, making sure it is nothing abusive, and then find ways of reducing it.
Thanks for finding this.
Unlimited rarely is.
Looks like backblaze (other post below) has a free bandwidth and cheap storage solution
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#138Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#139That reminds me of the entertaining "I just want to serve 5 terabytes. Why is this so difficult?" video that someone made inside Google. It satirizes the difficulty of getting things done at production scale. https://www.youtube.com/watch?v=3t6L-FlfeaI
I thought that was making fun of how difficult to implement anything at Google.
Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth
#140I helped run a wireless research data archive for a while. We made smaller data sets available via internet download but for the larger data sets we asked people to send us a hard drive to get a copy. Sneakernet can be faster and cheaper than using the internet. Even if you wanted to distribute 10TB of _new_ data every month, mailing hard drives would probably be faster and cheaper, unless all your customers are on I…
Nothing beats the bandwidth of a station wagon full of hard drives:-)