Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Canva saves Amazon S3 costs

canva.dev

Post reply on HN
Print view View original

51–60 of 181 posts

Re: How Canva saves Amazon S3 costs

#51

Post by withinboredom »

jaza wrote: post #48

I'm really surprised how much anti cloud opinion is being voiced here (but maybe I should have known better, after all these years on HN). Even if you don't use any other cloud tech in your stack, my advice is, use S3 (or equivalent)! That level of durability, availability, and scalability, is not trivial to pull off, and is best made somebody else's problem. Even when you're as big as Canva.

I'm a big fan of Garage[1], which is a dead-simple S3 drop-in that you can host on your own drives. It's designed for consumer hardware with shitty internet in-between nodes.

[1]:https://garagehq.deuxfleurs.fr/

Re: How Canva saves Amazon S3 costs

#52

Post by CogitoCogito »

maccard wrote: post #46
CogitoCogito wrote: post #16

Earlier quoted context omitted.

^^^ The cloud vs. on-prem argument often seems to ignore the (enormous) middle-ground. Just because one portion of your architecture would do well to be run outside the cloud, doesn't mean you take out all the other parts you don't want to deal with yourself. Furthmore, "on-prem" might mean in your building, in someone else's building co-located and you rent space and control it, or in someone else's building where t…

My team uses AWS for _everything_. There's three reasons for this on our team (and I've researched it for our use cases. - Consistency. Instead of saying "Oh look in AWS for this app, Azure for this one, and hetzner for this app, except it's test env is in AWS", it all just lives in AWS. It massively simplifies docs, onboarding, and reduces the amount of one-person specialised knowledge. - Engineering Costs. Similar…

> My team uses AWS for _everything_...If I started over, I'd make the same decisions again.

Okay? I never said that sticking to a single cloud provider isn't appropriate for some (or maybe even most) people. It's good that you have a setup that you believe works well for you.

Re: How Canva saves Amazon S3 costs

#53

Post by jlokier »

aljarry wrote: post #22
acchow wrote: post #5

Earlier quoted context omitted.

10 Petabytes is 10,000 TB. That's only 454 spinning disk drives (22 TB each). Approximate this as 600 with Raid 5. You can easily fit this in 100 desktops. There's tons of OSS to manage all layers of this stuff now.

But that's on a single RAID in a single location. You'd probably need at least one failover, and backups for that as well, if you'd like to target higher availability / durability. S3 keeps multiple copies in multiple geographic locations.

When you are distributing over multiple geographic locations, and much of the data can tolerate a longer round trip time for access, you can extend the RAID pattern across multiple locations, instead of within a location. Provided you have enough network bandwidth for recovery, you don't need a full RAID at each individual site.

Having more locations is actually helpful for this. Because of the independence of different locations, the risk of data loss is in some ways lower than a local RAID system where all drives might be fried at the same time by a single event.

Re: How Canva saves Amazon S3 costs

#54

Post by chii »

onion2k wrote: post #25
andrewstuart wrote: post #20

Earlier quoted context omitted.

>> moving from AWS to onprem means you need to configure the infrastructure yourself; you save on AWS fees but you spend more on devops This is the traditional “sell” for cloud computing and I don’t buy it at all. The clouds would have you believe it doesn't make sense to run your own systems because you need too much specialist expertise to run your own systems. The clouds sales pitch is the if you go cloud then you…

Don’t believe it. You don't need to 'believe'. You need to do an analysis of what you need and how much each option will cost. Then you can know . If your argument is "I believe onprem saves money" or "I bet AWS is cheaper" or "Jim Morrison came to me in a dream and said I should use Azure" then you haven't done enough research.

> you haven't done enough research.

the research isn't free either. The more indepth and time it takes to do such research, the slower you come to a decision and ship.

Cloud allows you to ship fast. It allows you to go without research - just accept the marketing, and pay up.

You pay above-cost (compared to on-prem) when you grow to a certain size. But this is usually a worthy trade off tbh.

Re: How Canva saves Amazon S3 costs

#55

Post by simonjgreen »

buildbot wrote: post #23
simonjgreen wrote: post #13

Earlier quoted context omitted.

Slight tangent from the point, however: friends don't let friends use RAID5. Putting this out as a PSA reminder as it's not often discussed any more given the prevalance of cloud storage. RAID5 gives you really the worst of all worlds. You get poor performance (n/4) due to the number of operations required and you get the fragility of a RAID storm that comes from a cascade RAID failure when a drive dies. A "stripe of…

Or “software” raid 10 with ZFS mirror dev’s all slapped into one pool?

Precisely that

Re: How Canva saves Amazon S3 costs

#56

Post by citrin_ru »

motoxpro wrote: post #42

A lot of people in this thread should start consultancies that transition companies off the cloud. Apparently, it would be easy for them to save tens of millions, in turn making millions for the consultant.

Consultants who help to move to the cloud are not cheap too. And migration to AWS/Azure is still more common than out.

Re: How Canva saves Amazon S3 costs

#58

Post by whycombagator »

andrewstuart wrote: post #2

Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…

What happens when cloudflare changes the price?

Re: How Canva saves Amazon S3 costs

#59

Post by ttymck »

andrewstuart wrote: post #2

Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…

Why do they need to get everything out? Some not-insignificant percentage of the content is surely archived/unused/unneeded assets. From customers who will never log in again or projects which are long over. Export the most recent 10% and put anything new on

Re: How Canva saves Amazon S3 costs

#60

Post by sofixa »

re-thc wrote: post #31
andrewstuart wrote: post #2

Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…

> Cloudflare R2 charges zero egress fees. The scarier part is Canva actually does use Cloudflare CDN, so it's not so much the cost of adding a new provider. It's just that the CDN team that manages the Cloudflare account is likely a different team :) Too much isolation, silos and politics...

The much more likely answer is that R2 is too new compared to when they started (announced in 2021) and switching from S3 to it is probably not trivial at Canva's scale.
Post reply on HN