Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

blog.cloudflare.com

Post reply on HN
Print view View original

191–200 of 220 posts

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#191

Post by tomnipotent »

loxias wrote: post #182
OJFord wrote: post #178

Earlier quoted context omitted.

'bandwidth alliance' As far as I recall it's not a service or even any setting you have to set up, just a sort of policy.

"Bandwidth Alliance" seems to be some sort of B2B consortium. I'll dig into this more later, but unless I'm missing something obvious (I very well might be...) there's not a easy/inexpensive method for me to sign up and join the "bandwidth alliance", so that data transfer from B2 to my laptop is free. I have a few VPSs with Linode, which is a member of the "Bandwidth Alliance" but I don't see any details, numbers, pr…

> there's not a easy/inexpensive method for me to sign up and join the "bandwidth alliance"

Not unless you run a data center and intend to peer with other bandwidth providers to join your intranet to the internet. It's intended for large service providers like Cloudflare/Backblaze that do direct bandwidth interconnect at physical locations and don't have to involve other intermediaries (like Level 3) to move data between members.

Otherwise you "join" by hosting services/content with an Alliance member and making sure you only use other services that do the same. Even then, bandwidth isn't always free (Azure and GCP still have some costs, for example, but discounted).

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#192

Post by maxgashkov »

neom wrote: post #189

I'm not a software developer so please pardon my ignorance, doesn't this put them at basically a "core complete modern cloud provider" - if you fully bought into architecting against them, with their workers product and all, could you fully build and run for CF?

I think they're still PaaS and not IaaS. So yes, you can build fully-fledged app with them if you're willing to change your engineering paradigm and not use any of the usual arbitrary computing workloads.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#193

Post by itake »

neom wrote: post #189

I'm not a software developer so please pardon my ignorance, doesn't this put them at basically a "core complete modern cloud provider" - if you fully bought into architecting against them, with their workers product and all, could you fully build and run for CF?

You could probably build a lot on their infra and maybe small b2b startups could get by on their offering, but you can easily outgrow their current offering. Azure/GCP/AWS have significantly more tools that solve expensive problems (like ML hosting, events, SQL database, search, etc.).

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#194

Post by alberth »

greg-m wrote: post #105
alberth wrote: post #96

This is fantastic, I only need now a Cloudflare RDMS to run my entire business on Cloudflare. (Workers KV is great but there's a ton of times that you just need an actual relational database)

Durable Objects is as close as we come today, but we still have two innovation weeks left this year.

Really hope you’re a Cloudflare employee, and that’s a teaser :)

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#195

Post by dan15 »

jauer wrote: post #167
loxias wrote: post #166

Earlier quoted context omitted.

(stupid question) What does that get me? Does Cloudflare have cheap VMs I can rent with competitive prices? (including data egress fees?) For object storage that I play with, some clients are in the cloud, with most sitting at home behind residential internet.

(approximately) Cloudflare provides a proxy service that you'd use to access your B2 data from home or other cloud without paying for egress. They can do this because it costs almost nothing to move data between B2 and Cloudflare, and then from Cloudflare to almost anywhere. Moving data from B2 to most other places on the internet likely costs them more because Backblaze isn't in a position to negotiate adventagous p…

Note that you can't use a free Cloudflare account just for things like images, video and other binary files, as they'll suspend the account. It must be used primarily for a website, not content hosting. If you only want to use Cloudflare for files, you need a paid account.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#196

Post by seabrookmx »

prirun wrote: post #179
stavros wrote: post #165

Earlier quoted context omitted.

Does this mean "remotes that speak the S3 protocol", or "remotes that are S3"? The former would require S3 supporting induced requests (so it POSTed the data somewhere), the latter would require a "copy" operation on S3. I don't know which one is supported.

Server side copy only works within the same service provider, not across service providers.

Right because, how would that work otherwise?

Unless those providers natively supported this functionality (and they're incentivized to not) you'd _need_ an intermediary.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#197

Post by throwaway984393 »

greg-m wrote: post #69
numair wrote: post #58

There is one major reason S3 remains the king of storage for mobile media uploads: bucket notifications. Does R2 implement this feature? If so, I’m going to have to run some experiments with this...

Hey, PM for R2 here! We're fully integrated with Workers, so you can write a Worker that calls additional logic when a request is made to the bucket's url. We have notifications, where a change to the bucket invokes a worker with a specific event, on our roadmap, but have a bunch of other work prioritized ahead of them.

I know this is irrational and not very helpful, but it actually makes me angry when I see a response like "but you can cobble together some janky custom code with Workers to do what you want".

It's like going to a restaurant, asking if they have shrimp scampi, and getting a reply that you can go to a supermarket and buy the ingredients and make the dish and bring it back to the restaurant to have with your meal.

Just tell me you don't have shrimp scampi.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#198

Post by OJFord »

loxias wrote: post #182
OJFord wrote: post #178

Earlier quoted context omitted.

'bandwidth alliance' As far as I recall it's not a service or even any setting you have to set up, just a sort of policy.

"Bandwidth Alliance" seems to be some sort of B2B consortium. I'll dig into this more later, but unless I'm missing something obvious (I very well might be...) there's not a easy/inexpensive method for me to sign up and join the "bandwidth alliance", so that data transfer from B2 to my laptop is free. I have a few VPSs with Linode, which is a member of the "Bandwidth Alliance" but I don't see any details, numbers, pr…

> there's not a easy/inexpensive method for me to sign up and join the "bandwidth alliance"

As I said, you don't have to. As far as I recall you just use two 'B's in the 'B2B consortium' and it's policy applies to you.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#199

Post by drip-egg-jack »

js4ever wrote: post #118
greg-m wrote: post #67

Earlier quoted context omitted.

Hey, PM for R2 here. We're still deciding on how we want to handle caching. We integrate with Workers, so manually caching is always possible. The catch is we're currently building for strong consistency - if we added a cache in front, we'd weaken that - so it will likely be a configuration option we add later.

I'm more interested in eventually consistent with low latency. Please add this setting ASAP :)

Check tebi.io - it is a geo-distributed S3-compatible storage that does exactly that. You can configure global consistency level for each bucket individually.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#200

Post by drip-egg-jack »

lewisl9029 wrote: post #171

Wow, with global replication by default, this looks absolutely perfect for what I'm currently building, even before taking costs into account. I'm hoping this means what I think it means, that write latencies will be minimal across the globe, since writes will be persisted and ack'd at the closest region and then eventually consistently propagated to other regions? If so, curious what would happen in a scenario where…

R2 is still under development, we will see how it will work, but I can tell you how Tebi.io global replication works:

- Writes and reads always goes to the closest region, this keeps network latency low. You can define different metadata write concern levels for each bucket, this way you can define how fast or how consistent your writes will be on a global scale. You can even make them asynchronous, meaning that once the data transfer is complete, metadata write and propagation is performed without you waiting for it to complete.

- If you write to one region and someone is trying to read that object from another region - it will be possible as soon as metadata is replicated to that region (usually it takes less than 500ms). If data is not yet replicated to that region, then it will be read from another region. If the data is partially replicated, then that part will be read from the local storage and the rest of the data from another region. Additionally, Tebi supports synchronous replication allowing almost instant data propagation across the world.

- Write latency depends on metadata replication write concern - it can be faster than AWS or slower, you can configure it yourself.

- You can define where you want to store your data and how many copies you need in each region.

Post reply on HN