Live data from Hacker News

A New Hope for Object Storage: R2 Enters Open Beta

blog.cloudflare.com

11–20 of 39 posts

Re: A New Hope for Object Storage: R2 Enters Open Beta

#11

I’m an engineer on the R2 team. Happy to answer any questions in this thread.

Any advice/guides on how to get this working with Rails ActiveStorage?

Should be the same as any other S3 backend. We don’t have examples for ActiveStorage per se but we have boto3, AWS JS SDK, and rclone [1].

[1] https://developers.cloudflare.com/r2/platform/s3-compatibili...

Re: A New Hope for Object Storage: R2 Enters Open Beta

#12

I’m an engineer on the R2 team. Happy to answer any questions in this thread.

1. TFA goes R2 clients will be able to enable caching (in the near future). A couple questions:

1a. Cloudflare Cache is free for users (up to 500MB per file for unlimited files), but if R2 uses it transparently, are reads from cache metered per R2 pricing? And what about the file-size limits that apply to free/pro accounts?

1b. How is strong-consistency guaranteed despite caches in front of the bucket? I guess cache-invalidation can be done through Durable Object (DO) Alarms or some such? Or, would this guarantee no longer hold?

2. TFA mentions in the passing that R2 utilizes DO (for presumably metadata / journal?). Cloudflare once wrote in detail about how DO is used in a pretty novel way to power wrangler tail, and so I was wondering if there's something novel about R2's use of DO, too?

3. In-Worker API: Are customers metered for both In-Worker API (duration and invocation) and R2 (Class A / Class B)? If so, that makes Class B operations almost twice as expensive than S3 (excluding egress).

4. Any ETA on "Public buckets"?

5. Has Cloudflare rolled out newer server designs just for R2 (aka Dropbox's MagicPocket)?

6. Is serving media content (say, audio and video) an acceptable use-case for R2?

7. Any plans for Kinesis Firehose (a managed-ingestion pipeline) esque product to front R2?

Thanks.

Re: A New Hope for Object Storage: R2 Enters Open Beta

#13

Neat! cl = boto3.client( 's3', aws_access_key_id=access_key_id, aws_secret_access_key=secret_access_key, endpoint_url=endpoint, config=Config( region_name = endpoints[endpoint_name].get('region', 'auto'), s3={'addressing_style': 'path'}, retries=dict( max_attempts=0 ), ), ) dict "endpoints" is undefined in the example code above, replace that line with: region_name = 'auto', ---- While, an S3-compatible API, which ex…

Yup. Sorry about that. I have a complicated Boto3 script I reduced but didn’t test and typo on the URL name.

We’re fixing the blog post.

Re: A New Hope for Object Storage: R2 Enters Open Beta

#14
post #2

We've been waiting for this since it was announced. We have > 2PB of media heavy data, and transfer a lot via S3 to the internet. This will reduce our monthly S3 charge from tens-of-thousands USD per month, to ~$200.

Please post a reply here when you get the "you're using too much bandwidth" email and the price that they'll quote you.

Thats problem is for CDN product. On R2 like other Developer products (KV, Worker, DO) you are already paying the bandwidth over the storage price. All those products can be used to HTML and non-HTML product.

https://www.cloudflare.com/supplemental-terms/#cloudflare-de...

Thats the major problem of the CDN product, they dont say that are no traffic fee, they just say they will not charge you "except" on some cases (that you never know). But other products they are very specific of the egress fee, like Stream, Magic Transit, Spectrum, etc.... They should fix that matter on CND, no one like that.

Sure, i dont know what will happen if someone just try to make a youtube like over R2... They probable "want" that....

Re: A New Hope for Object Storage: R2 Enters Open Beta

#15
post #14

Earlier quoted context omitted.

Please post a reply here when you get the "you're using too much bandwidth" email and the price that they'll quote you.

Thats problem is for CDN product. On R2 like other Developer products (KV, Worker, DO) you are already paying the bandwidth over the storage price. All those products can be used to HTML and non-HTML product. https://www.cloudflare.com/supplemental-terms/#cloudflare-de... Thats the major problem of the CDN product, they dont say that are no traffic fee, they just say they will not charge you "except" on some cases (t…

This is the exact case when the "except" will be triggered. He is describing a video/image cdn in his comment.

Re: A New Hope for Object Storage: R2 Enters Open Beta

#17

I’m an engineer on the R2 team. Happy to answer any questions in this thread.

1. TFA goes R2 clients will be able to enable caching (in the near future). A couple questions: 1a. Cloudflare Cache is free for users (up to 500MB per file for unlimited files), but if R2 uses it transparently, are reads from cache metered per R2 pricing? And what about the file-size limits that apply to free / pro accounts? 1b. How is strong-consistency guaranteed despite caches in front of the bucket? I guess cach…

1a. I don't think we have anything to share with regards to the logistics of putting cache in front of R2 or how that product will work.

1b. Unavoidably as you guess you'd be giving up the consistency. Presumably the consistency guarantees would be part of the cache-control header you associate with the object. Cache purging would probably work similar & we have exciting upcoming news about major performance improvements for that. But again, since we don't have any idea of what that product looks like, hard to say.

2. There is. I have a bunch of technical blog posts written but we decided to punt on providing that detail for a few months just to give everyone some breathing room. R2 does have one important extension we got the DO team to build just for us (not 1 instance of the code instantiated). That piece will only be available for R2. R2 is also driving performance improvements & API improvements to DO that will almost certainly percolate out to external users.

3. The equivalent model would be AWS Lambda + S3 so I think we're significantly cheaper (+ Workers is faster than AWS lambda - R2 is still in open beta so not useful to compare perf against S3 yet). Yes, if you're using it to emulate public buckets, then it'll be a bit expensive (depending on egress charges you might otherwise face)

4. Soon. It's very high on our list alongside pre-signed URLs

5. Nothing to share at this time.

6. Yes. From Matthew Prince aka eastdakota [1]

7. Nothing to share at this time.

[1] https://news.ycombinator.com/item?id=28682885

Re: A New Hope for Object Storage: R2 Enters Open Beta

#19
post #18

Amused by the Star Wars reference in the title. (D1 should have been called D2, just to get the complete name).

Yeah, but that could be another sequel. D1 is one minus D2

https://twitter.com/eastdakota/status/1524392750342238214?cx...

Re: A New Hope for Object Storage: R2 Enters Open Beta

#20
post #2

We've been waiting for this since it was announced. We have > 2PB of media heavy data, and transfer a lot via S3 to the internet. This will reduce our monthly S3 charge from tens-of-thousands USD per month, to ~$200.

The storage cost on r2 is .015/GB. Might want to redo the math
Post reply on HN