> R2 is an object store. It is literally a drop-in replacement to S3.
A CDN is a thing where you can put things, and request them by web, and have them geo-distributed.
R2 is actually a CDN. So is S3. Yes, I'm aware, the vending companies also sell different products called CDNs. Nonetheless, R2 and S3 (unlike B2) are CDNs. You can make distributed hits to it across the planet to addresses you don't control. You can hit it directly from the web, and it will serve to you from a local node you never knew anything about. Its goals are speed and locality. You can look up programmers debating whether to use CloudFront or S3 as their CDN based on time to invalidate vs actual speed and cost control.
Attempting to describe it as "an object store instead of a CDN" is kind of missing the point. It's both. It's also a webserver. It's also a backup system. It's many things.
You cannot replicate R2 on a single machine. R2 delivers locality and redundancy. Arguing over its title is irrelevant; the reason to point out that it is a CDN is to establish requirements for replacing it with a homebrew solution.
The lowest practical cost for reliable machines is arguably VPSes at around two dollars a month. Those VPSes tend to give you 100-250 megabits of traffic monthly with around 1gig of storage. Assuming two for redundancy, you'll get maybe 500 megabits for about $4 a month. (I personally would not feel safe with two, but we're talking about cost cutting.)
That same 1g of storage on R2 falls in the free forever tier.
To eat up the $4 a month in class B actions, you will first need to consume the 10 million free events, then spend 36 cents per further million. This is a further 11.1 million, or 21.1 million requests per month.
Most low-tier VPSes will struggle around 10 requests per second using NGINX, because their disks and memory are terribly over-burdened. There are 86,400 seconds in a day, suggesting you'll get around 860,000 requests per day if you don't consider time shaping. Considering time shaping - you don't have full requests coming in at every time zone - you'll most likely have closer to 350,000 realistic. This means that your two VPSes will, just barely, be able to close the same amount of traffic.
Very, very slowly. You'll be lucky to get 200ms responses locally, and to get VPSes that cheap, you'll need one in the Netherlands in the burden corridor, and one in the United States, most likely in Texas or Georgia.
Most of your customers will now be getting 300ms round trips.
Why? Because you wanted to save against four dollars a month. This isn't even enough money to biggie-size a value meal.
All so that you could make a two-computer CDN on VPSes.
In the meantime, I've worked at two unicorns and neither of them had traffic anywhere near this large.
You can also do the math with an unmetered 1u. Ten megabit unmetered with 95% availability typically goes for about $19 a month, or $30 a month with a cheap 1u attached. That's about 3.2t of traffic a month.
As soon as you try to put actual specific numbers to it, and calculate actual specific costs, this just falls apart.
.
> There are at least 10 (if not more) CDN providers out there that have footprints in several thousand datacenters
There's more than two thousand of them from the United States alone. Akamai's CDN has over 400,000 nodes at over 50,000 POPs.
I can think of more than 40 in-house just from the US of that size (eg Netflix, Steam.)
What's your point?
.
> R2, B3, and Wasabi entering the game
Wasabi has been around since 2015. B2 isn't really the same kind of thing as R2 or S3.
.
> because CDNs are a competitive market ... Object stores are becoming a competitive market
Object stores have been around decades longer than CDNs.
.
> but buying your own machines and using Ceph is still cheaper.
Not really, no. I just did the numbers, and they did not pan out the way you claim.
Buying your own machines? You're lucky to find a 1u for less than $1500, and colo for a 1u is typically at least $40/mo. That covers 30 million writes and 300 million reads a month for six months at R2. (For scale, the Washington Post gets 86 million uniques a month, so if WaPo is doing four hits per load, which seems high if you sprite your images, then you're talking about larger than WaPo traffic. Spriting your images is a lot less work than building a CDN from scratch.)
The R2 cost for Washington Post is about $230 a month. You ... you really want to fight a cost like that, at a scale like that?
At that price, you're saving about $190 a month on bandwidth, so disregarding the cost of upkeep, it takes about eight months of WaPo tier traffic per slab to break even on each $1,500 slab.
In order to compare against R2 you need multiple locations (redundancy is critical to safety,) so for this to make sense, you need to be able to deliver multiple Washington Posts of traffic. If you can do that, well, I can make $230 a month not a big deal to you. Let's talk.
If you're trying to save money against four dollars a month, which I think was the actual realistic price here, I'd warrant it would be a good idea to look at your salary, convert it to hourly, and figure out how many years you'd have to run that savings to recoup your first hour of thinking about it. (Heck, I would even say that about $230 a month - after all, if you make $125k that's about an hour of your time, and I doubt even a very talented engineer could set up a distributed R2 replacement, including buying and siting machines, that fast.)
The vertical step cost is larger than the total cost in almost any practical system.
In order for the self-hosting costs to pan out against a distributed system like R2, you need to be looking at weekly terabytes of traffic and hundreds of nodes.
Respectfully, no, doing this yourself isn't actually cheaper. Not even close. You'll need a cost outlay to convince me, and it'll have to explain why the one I just did is wrong.
A realistic "buy your own hardware" CDN^H^H^Hobject store doesn't have barebones 1us with tiny drives. A realistic object store has a couple attached JBODs. (But you know, this is some individual's personal file server, which has been lionized into being a "competitor to R2" by ignoring almost everything that R2 actually does.)
And of course, the second you actually pull that cost outlay off, I'm just going to start talking about how much cheaper it would be to build your own datacenter, and then your own internet. Because as long as we're not focusing on there being no business need to justify the outlay, the larger you build, the better.
But it's "object hosting," not an R2 competitor, and you want to self-build. So maybe we just knock this down to two nodes. And then it's only $4,200 up front and $1,700 of traffic a month to break even, plus engineer costs for building, maintaining, and debugging it.
(And honestly, if you're just using it for storage, why not just set up FTP or RCP?)
So I googled "how much traffic should your website get." I got hubspot. https://blog.hubspot.com/blog/tabid/6307/bid/5092/how-many-v...
That article - and I'm not entirely certain that authoritative even exists here if someone defines things, which I haven't - but that article is probably better to listen to than me, at least.
They claim they surveyed about 400 "traffic analysts." There's a common sense line here of "they work for people who can afford entire staff for a job like that," so I guess I assume by default that these folks mostly work for very large sites, and the remainder merely for large sites.
Like I don't think any private pages with 200 hits a month have a "traffic analyst," you know?
Sure enough, even in that survey, 46% of them were in the 1k-15k a month bucket, and less than half a percent were in the 10m+ bucket. And remember, we needed 300 million per server-month to break even.
My strongly held, evidence based opinion is that the scale at which CDNs are cost-competable is fundamentally an irrelevant scale to all but the very largest of sites, in the way that a local single-location restaurant should not be looking into making its own flatware or furniture to control overhead.
And none of this counts the engineer costs.
My opinion is that in order to cost compete against CDNs, you need to have hundreds of large websites as customers.
My opinion is that viewing R2 and S3 as "just a place to store files" is akin to viewing a car as a place to listen to music - not even the primary use case. S3 has had http access from day one.
And yes, you can make a far cheaper radio than a car. But in general, if you try to sell the result to car purchasers, what I believe you will find is that you've misunderstood what the market is attempting to purchase.
My experience is that 90% of the people I know using S3 are using it for web fronting directly, or to back CloudFront.
.
Null hypotheses are important. So is actually having the ability to use what you build.
Imagine how cheap I could make the CDN if you just keep adding zeroes to the userbase all day, right? But it would be a hollow demonstration, because I can't actually deliver the customers to justify it.
The reason I've never heard of Ceph is that nobody I know uses it. Nobody I know uses it because nobody I know would invest this much effort into getting rid of a couple dollar a month datastore.
I know thousands of people who are making SAASes and other sites.
I am of the opinion that recreating object stores and CDNs is one of the smallest levers that you can think about turning, unless you're some starkly uncommon kind of site like a video host.