Live data from Hacker News

From S3 to R2: An economic opportunity

dansdatathoughts.substack.com

61–70 of 183 posts

Re: From S3 to R2: An economic opportunity

#61
>> you’re paying anywhere from $0.05/GB to $0.09/GB for data transfer in us-east-1. At big data scale this adds up.

At small data scale this adds up.

And..... it's 11 cents a GB from Australia and 15 cents a GB from Brazil.

If you have S3 facing the Internet a hacker can bankrupt your company in minutes with simple load testing application. Not even a hacker. A bug in a web page could do the same thing.

Re: From S3 to R2: An economic opportunity

#62

I wish the R2 access control was similar to S3 - able to issue keys with specific accesses to particular prefixes, and ability to delegate ability to create keys. It currently feels a little limited and… bolted on to the Cloudflare UI.

I think the idea is to use Cloudflare Workers to add more sophisticated functionality.

Re: From S3 to R2: An economic opportunity

#63

Is there any reason to not use R2 over a competing storage service? I already use Cloudflare for lots of other things, and don't personally care all that much about the "Cloudflare's near-monopoly as a web intermediary is dangerous" arguments or anything like that.

As far as I know, R2 offers no storage tiers. Most of my s3 usage is archival and sits in glacier. From Cloudflare's pricing page, S3 is substantially cheaper for that type of workload.

Re: From S3 to R2: An economic opportunity

#64

Is there any reason to not use R2 over a competing storage service? I already use Cloudflare for lots of other things, and don't personally care all that much about the "Cloudflare's near-monopoly as a web intermediary is dangerous" arguments or anything like that.

1. This is the most obvious one, but S3 access control is done via IAM. For better or for worse, IAM has a lot of functionality. I can configure a specific EC2 instance to have access to a specific file in S3 without the need to deal with API keys and such. I can search CloudTrail for all the times a specific user read a certain file.

2. R2 doesn't support file versioning like S3. As I understand it, Wasabi supports it.

3. R2's storage pricing is designed for frequently accessed files. They charge a flat $0.015 per GB-month stored. This is a lot cheaper than S3 Standard standard pricing ($0.023 per GB-month), but more expensive than Glacier and marginally more expensive than S3 Standard - Infrequent Access. Wasabi is even cheaper at $0.0068 per GB-month but with a 1 TB billing minimum.

4. If you want public access to the files in your S3 bucket using your own domain name, you can create a CNAME record with whatever DNS provider you use. With R2 you cannot use a custom domain unless the domain is set up in Cloudflare. I had to register a new domain name for this purpose since I could not switch DNS providers for something like this.

5. If you care about the geographical region your data is stored in, AWS has way more options. At a previous job I needed to control the specific US state my data was in, which is easy to do in AWS if there is an AWS Region there. In contrast R2 and Wasabi both have few options. R2 has a "Jurisdictional Restriction" feature in Beta right now to restrict data to a specific legal jurisdiction, but they only support EU right now. Not helpful if you need your data to be stored in Brazil or something.

Re: From S3 to R2: An economic opportunity

#65
post #16

Earlier quoted context omitted.

Also, for the CDN case that R2 seems to be targeting - regardless of the origin of the data (R2 or S3), chances are pretty good that Cloudflare is already paying for the egress anyway.

I'm not sure about that. A CDN keeps the data nearby, reducing the need to pay egress to the big bandwidth providers. ( not an expert though)

Let's say you want to use cloudflare, or another CDN. The process is pretty simple.

You setup your website and preferably DON'T have it talk to anyone other than the CDN.

You then point your DNS to wherever the CDN tells you to. (Or let them take over DNS. Depends on the provider.)

The CDN then will fetch data from your site and cache it, as needed.

Your site is the "origin", in CDN speak.

If Cloudflare can move the origin within their network, there is huge cost savings and reliability increases there. This is game changing stuff. Do not under estimate it.

Re: From S3 to R2: An economic opportunity

#66

Cloudflare has been attacking the S3 egress problem by creating Sippy: https://developers.cloudflare.com/r2/data-migration/sippy/ It allows you to incrementally migrate off of providers like S3 and onto the egress-free Cloudflare R2. Very clever idea. He calls R2 an undiscovered gem and IMO this is the gem's undiscovered gem. (Understandable since Sippy is very new and still in beta)

Clever

Re: From S3 to R2: An economic opportunity

#67

Cloudflare has been attacking the S3 egress problem by creating Sippy: https://developers.cloudflare.com/r2/data-migration/sippy/ It allows you to incrementally migrate off of providers like S3 and onto the egress-free Cloudflare R2. Very clever idea. He calls R2 an undiscovered gem and IMO this is the gem's undiscovered gem. (Understandable since Sippy is very new and still in beta)

What are the economics that Amazon and other providers have egress fees and R2 doesn't? Is it acting as a loss leader or does this model still make money for CloudFlare?

There has to be more to it than a pure loss leader, since there's also the Bandwidth Alliance Cloudflare is in, which allows R2 competitors like Backblaze B2 to also offer free egress, which benefits those competitors while weakening the incentive for R2 somewhat.

Re: From S3 to R2: An economic opportunity

#68

Is there any reason to not use R2 over a competing storage service? I already use Cloudflare for lots of other things, and don't personally care all that much about the "Cloudflare's near-monopoly as a web intermediary is dangerous" arguments or anything like that.

If you already use Cloudflare for lots of other things, no.

If you already use AWS for lots of other things, yes.

Re: From S3 to R2: An economic opportunity

#69
post #25

Earlier quoted context omitted.

I think the subtle API differences reflect bigger and deeper implementation differences... For example, "Can one append to an existing blob/resume an upload?" leads to lots of questions about data immutability, cacheability of blobs, etc. "What happens if two things are uploaded with the same name at the same time" leads into data models, mastership/eventual consistency, etc. Basically, these 'little' differences are…

This is a good point, but just a standard for the standard create/read/update (replace)/delete operations combined with some baseline guarantees (like approximately-last-write-wins eventual consistency) would probably cover a whole lot of applications that currently use S3 (which doesn't support appends anyway). Heck, HTTP already provides verbs that would cover this, it would just require a vendor to carve out a sub…

[deleted]

Re: From S3 to R2: An economic opportunity

#70

>> you’re paying anywhere from $0.05/GB to $0.09/GB for data transfer in us-east-1. At big data scale this adds up. At small data scale this adds up. And..... it's 11 cents a GB from Australia and 15 cents a GB from Brazil. If you have S3 facing the Internet a hacker can bankrupt your company in minutes with simple load testing application. Not even a hacker. A bug in a web page could do the same thing.

200 TB in minutes is impressive.

(Assuming your company can be bankrupted for ~$20k.)

Post reply on HN