Live data from Hacker News

Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

blog.cloudflare.com

151–160 of 220 posts

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#152
post #106

This is interesting ... For the longest time we tried to convince people that they should have an off-amazon archive of their S3 data ... we even ran an ads to that effect in 2012[1]. The (obvious) reason this isn't compelling is the cost of egress. It's just (relatively) too expensive to offload your S3 assets to some third party on a regular basis. So if R2 is S3 with no egress, suddenly there is a value propositio…

> 'rclone'[2][3] which allows you to move data from cloud to cloud without involving your own bandwidth

Maybe I'm reading this wrong, but the data does pass through the machine where rclone is running. rclone does support remote-to-remote transfers[0], but I believe only for remotes of the same type (ie S3 to S3).

[0]: https://rclone.org/docs/#server-side-copy

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#153

Earlier quoted context omitted.

I personally think uploading massive files is not usually desirable. Better would be an easy way to chunk it and upload and have the server put the file back together, which would increase reliability.

Native tus support would be pretty amazing.

Tus is over-complicated IMO. Why require the server to store state that's already available on the file system? Just support PATCH requests with an offset parameter. If an upload fails partway through, HEAD the file to see how much got copied to disk and resume from there.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#154
post #123

Very generous offer from Cloudflare. I signed up. The main question: how can Cloudflare make this into a sustainable business? * cost/gb is cheaper or same as s3, gcp, azure * no egress charges to customers, but they still have to pay for transit when they cross an AS! what is the hidden angle Cloudfare is using here?

Cloudflare is already providing free egress to object storage hosted elsewhere (like B2). Now they can handle the storage too and charge you for it.

It's a net gain, with a storage price higher than B2 or Wasabi, and also moves your data to their platform for more ecosystem lock-in.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#155

Nice to see someone flipping the script and encroaching on AWS' territory rather than vice a versa Taking the have-a-much-better-product route to siphoning use from AWS is particularly ambitious. I hope it works out. AWS has had it a little too easy for too long

Actually many companies are trying/tried to compete with AWS especially in an egress price territory but because of a lack of exposure it is very hard to get a traction and get big enough.

And there is a lot of things that can be improved - like egress price, better support, get rid of regions completely, etc...

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#157
post #99

Earlier quoted context omitted.

What format, bitrate, and resolution are your video outputs? Or just a length and file size of one that’s handy. (I’m a streaming video engineer and curious.) Reduction in file size from re-encoding doesn’t mean there will be perceivable loss in quality. Your videos should almost certainly not require the kind of bandwidth that you mention unless your deliverables are higher-quality than those from, say, Netflix :) F…

No worries, I've always been more interested by the technical aspect more than the creative aspect of filmmaking, hence my interest in programming. It's a personal project, I've been making films since the age of 11, although I haven't been able to do much since I started my Bachelor degree... I encode H.264/AAC for compatibility, usually 2-pass at 8Mbit/s for FHD content or around 30Mbit/s for UHD, 256-320 kbps for…

I suggest to use constant quality encoding instead of a constant bitrate, this way encoder will automatically adapt bitrate for a particular scene. This is a much better approach, it will give you better quality and a smaller file at the same time.

For example, encoder might choose 1Mbps bitrate for a scene which is mostly static and 20Mbps for a part of the video with a lot of movement. Your 8Mbps constant bitrate will be an overkill for the first scene and too low for the second. Let encoder decide the optimal bitrate.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#158
post #123

Very generous offer from Cloudflare. I signed up. The main question: how can Cloudflare make this into a sustainable business? * cost/gb is cheaper or same as s3, gcp, azure * no egress charges to customers, but they still have to pay for transit when they cross an AS! what is the hidden angle Cloudfare is using here?

Cost/GB is three times that of Backblaze, bandwith is generally too cheap to meter (outside aws, gcp, azure who use egress costs for lock-in). The offer is no doubt competitively prized, but it's no doubt much more lucrative than people using Cloudflare as a CDN for their s3/b2/azure/whatever.

You need to account IO load as well, more bandwidth means more requests to the disks and I assume they will use mechanical HDDs for the storage, SSDs are still too expensive to support $0.015/GB. Good mechanical drive can give you up to 200 IOPS and maybe 200MB/s, so they will have to copy the data a lot to be able to increase capacity on demand. Making it free doesn't seems to be sustainable at all.

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#159
I see the PM for this product is here, a few things we find useful with S3:

- Daily inventory report of the contents of the bucket in parquet format

- Signed URLs

- Lifecycle policies based on tags (but to be honest just a policy that isn't restricted to suffix/prefix would be amazing)

- Bucket policies (restrict object key structure)

Lots of these work well enough in AWS but are often lacking in some regards with annoying restrictions.

Looks like an amazing product, good luck!

Re: Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees

#160
>providing zero-cost egress for stored objects — no matter your request rate

What's the catch? Imagine a few cases. Let's assume s3 volume rate of $50/TB.

-I post 1gb video file on Reddit. 100k downloads / month: $5k

-I make a 1gb desktop app. I have 100k downloads / month: $5k

-I post 100gb data file on Github. 10K downloads / month: $50k.

Would I pay $0 on R2? And would there be throttling / rate-limiting?

[Edit: Added more realistic examples]

Post reply on HN