Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 199 posts

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#81

Post by winrid »

You could do this for about $1k/mo with Linode and Wasabi.

For FastComments we store assets in Wasabi and have services in Linode that act as an in-memory+on disk LRU cache.

We have terabytes of data but only pay $6/mo for Wasabi, because the cache hit ratio is high and Wasabi doesn't charge for egress until your egress is more than your storage or something like that.

The rest of the cost is egress on Linode.

The nice thing about this is we gets lots of storage and downloads are fairly fast - most assets are served from memory in userspace.

Following thread to look for even cheaper options without using cloudflare lol

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#82

Post by capableweb »

aborsy wrote: post #80

S3 is probably the highest quality. It’s enterprise grade : fast, secure with a lot of tiers and controls. If you recover only small data, it’s also not expensive. The only problem is if you recover large data. That would be a major problem.

10TB storage + 100TB bandwidth and S3 will easily be +1000 USD per month, while there are solutions out there that are fast and secure with unrestricted bandwidth for less than 100 USD per month. Magnitude cheaper with same grade in "enterprisey".

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#83

Post by codersfocus »

wongarsu wrote: post #17

Depends on what exactly you want to do with it. Hetzner has very cheap Storage boxes (10TB for $20/month with unlimited traffic) but those are closer to FTP boxes with a 10 connection limit. They are also down semi-regularly for maintenance. For rock-solid public hosting Cloudflare is probably a much better bet, but you're also paying 7 times the price. More than a dedicated server to host the files, but you get more…

I wouldn't call Cloudflare rocksolid, assuming you mean their R2 offering. It goes down pretty regularly.

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#85

Post by spiffytech »

mkroman wrote: post #45
callamdelaney wrote: post #41

Wasabi, unlimited bandwidth. $5.99/tb/month. Though it is object storage. See: https://wasabi.com/cloud-storage-pricing/#cost-estimates They could really do with making the bandwidth option on this calculator better.

Wasabi does not have any sort of unlimited bandwidth. In fact, they explicitly say that if your download exceeds 100TB/month your use case is not a good fit. https://wasabi.com/paygo-pricing-faq/#free-egress-policy

Wasabi also bills you for a minimum of 90 days of storage, so it's only appropriate if your data has low turnover.

https://wasabi.com/paygo-pricing-faq/#minimum-storage-durati...

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#86

Post by aborsy »

capableweb wrote: post #82
aborsy wrote: post #80

S3 is probably the highest quality. It’s enterprise grade : fast, secure with a lot of tiers and controls. If you recover only small data, it’s also not expensive. The only problem is if you recover large data. That would be a major problem.

10TB storage + 100TB bandwidth and S3 will easily be +1000 USD per month, while there are solutions out there that are fast and secure with unrestricted bandwidth for less than 100 USD per month. Magnitude cheaper with same grade in "enterprisey".

Well, I said, if you store small data. For large data, sure, prohibitively expensive!

I don’t think many other solutions are equally fast and secure.

AWS operation is pretty transparent, documented, audited and used by governments. You can lock it down heavily with IAM and a CMK KMS key, and audit the repository. The physical security is also pretty tight, and there is location redundancy.

Even hetzner doesn’t have proper redundancy in place. Other major providers in France burned down (apparently with with data loss), or had security problems with hard drives stolen in transport.

I don’t work for AWS, don’t have much data in there, just saying. GCP and Azure are probably also good.

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#87

Post by k8sToGo »

GC_tris wrote: post #79
k8sToGo wrote: post #74

People here suggest Hetzner. Just be aware that their routing is maybe not as good as you get for more expensive bandwidth.

Can you be more specific? Hetzner has excellent connectivity: https://www.hetzner.com/unternehmen/rechenzentrum/ They are always working to increase their connectivity. I'd even go so far to claim that in many parts of the world they outperform certain hyperscalers.

I used to have a dedicated server there and what happened to me is that my uploads were fast, but my downloads were slow. Looking at an MTR route, it was clear that the route back to me was different (perhaps cheaper?). With google drive for example I could always max out my gbit connection. Same with rsync.net

Also I know that some cheaper Home ISPs also cheap out on peering.

Now, this was some time ago, so things might have changed, just as you suggested.

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#89

Post by charcircuit »

aborsy wrote: post #86
capableweb wrote: post #82

Earlier quoted context omitted.

10TB storage + 100TB bandwidth and S3 will easily be +1000 USD per month, while there are solutions out there that are fast and secure with unrestricted bandwidth for less than 100 USD per month. Magnitude cheaper with same grade in "enterprisey".

Well, I said, if you store small data. For large data, sure, prohibitively expensive! I don’t think many other solutions are equally fast and secure. AWS operation is pretty transparent, documented, audited and used by governments. You can lock it down heavily with IAM and a CMK KMS key, and audit the repository. The physical security is also pretty tight, and there is location redundancy. Even hetzner doesn’t have p…

>Well, I said, if you store small data.

Well, the OP said he would be using >100 TB a month.

>GCP and Azure are probably also good.

They similarly charge 100x for bandwidth. No they are not a good option either.

Re: Ask HN: Suggestions to host 10TB data with a monthly +100TB bandwidth

#90

Post by novok »

wongarsu wrote: post #17

Depends on what exactly you want to do with it. Hetzner has very cheap Storage boxes (10TB for $20/month with unlimited traffic) but those are closer to FTP boxes with a 10 connection limit. They are also down semi-regularly for maintenance. For rock-solid public hosting Cloudflare is probably a much better bet, but you're also paying 7 times the price. More than a dedicated server to host the files, but you get more…

Hetzner is also fairly slow network bandwidth wise unless you're in Europe.
Post reply on HN