Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

blog.cloudflare.com

Post reply on HN
Print view View original

91–100 of 220 posts

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#91

Post by mnw21cam »

greg-m wrote: post #60
sudhirj wrote: post #44

I know the Cloudflare team hangs out here, so thanks, and great job! This was absolutely necessary for my line of work. Couple of quick questions/confirmations: * R2 will support the same object sizes as S3? We have 500GB+ objects and could go to a 1TB per object. * R2 will support HTTP Range GETs, right? Egress bandwidth for objects on S3 is the biggest line item on the AWS bill for a company I work for, by an order…

Hey, I'm the PM for R2. Yes to range requests. Current object limit is smaller than that, but we don't have a fundamental restriction there. Shoot me an email to gmckeon [at] cloudflare.

I'd say definitely increase that limit. I have a 4TB file hanging around over here, I'm sure others have bigger ones.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#92

Post by fideloper »

ignoramous wrote: post #88

> As transformative as cloud storage has been, a downside emerged: actually getting your data back... When they go to retrieve that data, they're hit with massive egress fees that don't correspond to any customer value — just a tax developers have grown accustomed to paying. Strategy Letter V, commoditize your competitor's advantages! > We’ve gotten rid of complex, manual tiering policies in favor of what developers…

Being able to append to an object would be amazing!

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#93

Post by dikei »

ryanworl wrote: post #8

Could you provide some more details on the storage system here? Is it built on Ceph's S3 compatibility? Your durability numbers imply erasure coding. Is that the case?

> Your durability numbers imply erasure coding. Is that the case?

Well, you can't really build inexpensive, large scale, reliable storage without erasure coding. So that's probably a given.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#94

Post by greg-m »

mnw21cam wrote: post #91
greg-m wrote: post #60

Earlier quoted context omitted.

Hey, I'm the PM for R2. Yes to range requests. Current object limit is smaller than that, but we don't have a fundamental restriction there. Shoot me an email to gmckeon [at] cloudflare.

I'd say definitely increase that limit. I have a 4TB file hanging around over here, I'm sure others have bigger ones.

we will :) we're working to reach 5 GB per upload, with multi-part support.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#95

Post by danielrhodes »

greg-m wrote: post #94
mnw21cam wrote: post #91

Earlier quoted context omitted.

I'd say definitely increase that limit. I have a 4TB file hanging around over here, I'm sure others have bigger ones.

we will :) we're working to reach 5 GB per upload, with multi-part support.

I personally think uploading massive files is not usually desirable. Better would be an easy way to chunk it and upload and have the server put the file back together, which would increase reliability.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#97

Post by zackbloom »

danielrhodes wrote: post #95
greg-m wrote: post #94

Earlier quoted context omitted.

we will :) we're working to reach 5 GB per upload, with multi-part support.

I personally think uploading massive files is not usually desirable. Better would be an easy way to chunk it and upload and have the server put the file back together, which would increase reliability.

Native tus support would be pretty amazing.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#98

Post by xrendan »

alberth wrote: post #96

This is fantastic, I only need now a Cloudflare RDMS to run my entire business on Cloudflare. (Workers KV is great but there's a ton of times that you just need an actual relational database)

Depending on your use-case (if it's read heavy), you could use SQLite hosted on R2 being read by workers

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#99

Post by dkh »

marcus_cemes wrote: post #34
corobo wrote: post #18

Earlier quoted context omitted.

Have you seen their Stream product, any use? Came out of their last batch of features if I recall https://www.cloudflare.com/en-gb/products/cloudflare-stream/

I have, I've also taken a look at Mux. Both would be fantastic options, and I'm still considering them, but I don't have many films and I'm biased towards distributing the original high-quality encodes. Both of these services significantly reduce the file-size with a re-encode, even if they promote an "impercievable quality loss". They seem to be more suited to high traffic on-demand streaming for websites, promotion…

What format, bitrate, and resolution are your video outputs? Or just a length and file size of one that’s handy. (I’m a streaming video engineer and curious.) Reduction in file size from re-encoding doesn’t mean there will be perceivable loss in quality. Your videos should almost certainly not require the kind of bandwidth that you mention unless your deliverables are higher-quality than those from, say, Netflix :)

FYI Mux isn’t an okayish platform for mid-level use-cases, it’s the gold standard for high-performance live streaming and VOD, created by the guys behind a bunch of other stuff pretty central to the streaming industry overall, and is used at the highest level by folks like CBS, Mandolin, etc. Of course you don’t have to use it, but it’s certainly no toy.

Just curious, does your audience expect to download it in full to watch later or on a USB stick as you describe, or is that a side effect due to the large file sizes?

EDIT: I am not a Mux employee, nor is any of this very important, I've just been working in this space for 10 years and very seldom run across something that needs these requirements and I'm curious :)

Post reply on HN