How Canva saves Amazon S3 costs
canva.dev
How Canva saves Amazon S3 costs
1–10 of 181 posts
Re: How Canva saves Amazon S3 costs
#2Seems crazy to run such large infrastructure on a major cloud.
Tens of millions wasted.
Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible.
S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave.
S3 is 9 cents per GB egress fees.
Cloudflare R2 charges zero egress fees.
At list price of 9 cents per GB it would cost Canva about USD$20 million to export their 230 petabytes. Let me know if my calculation is wrong.
Re: How Canva saves Amazon S3 costs
#3Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…
Yup, we just moved large chunks of our SaaS platform from the cloud to dedicated servers with Kubernetes serving as an internal cloud. Ended up reducing our cost to 30% (70% saved!).
We realized that instead of using scaling for peaks, by having additional dedicated servers available full time - it still worked out significantly cheaper. We also moved a lot of our internal processing to run in specific windows of time where the expected load was low to maximize server utilization in those periods.
Re: How Canva saves Amazon S3 costs
#4Real question, how to scale a business to petabytes of data without cloud? How is it done in practice?
Re: How Canva saves Amazon S3 costs
#5Real question, how to scale a business to petabytes of data without cloud? How is it done in practice?
10 Petabytes is 10,000 TB. That's only 454 spinning disk drives (22 TB each). Approximate this as 600 with Raid 5. You can easily fit this in 100 desktops.
There's tons of OSS to manage all layers of this stuff now.
Re: How Canva saves Amazon S3 costs
#6Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…
There's always a cheaper way to do something, but it's important to remember that 'lower price' often doesn't mean 'lower cost'. In order to get the lower price you need to spend on whatever the alternative option isn't doing in order to give that price saving. For example, moving from AWS to onprem means you need to configure the infrastructure yourself; you save on AWS fees but you spend more on devops. And then you have to factor in things like the cost of downtime (on both sides of the equation, especially if you use us-east-1), the price of building your own services, the price of software you need to buy, and so on.
You only save money if the differential cost based on all the factors is lower. AWS is expensive for some things so it often does save some money, but not always, and if you haven't done a proper analysis you can't know.
Re: How Canva saves Amazon S3 costs
#7Real question, how to scale a business to petabytes of data without cloud? How is it done in practice?
Buy hard drives. Typically the spinning ones for the data because $/GB is really cheap. Then you buy some SSDs for your actual applications to run on.
From there you install Hadoop, maybe a query tool like Spark, Impala, and Presto. Then you install some reporting tools like Apache SuperSet.
At least that's how we did it at my old job with Petabytes of data.
Re: How Canva saves Amazon S3 costs
#8Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…
Many aws services are hit or miss but s3 is just a marvel. Running geo-distributed 100s PB Ceph cluster with tiered storage isn't exactly trivial.
Re: How Canva saves Amazon S3 costs
#9Real question, how to scale a business to petabytes of data without cloud? How is it done in practice?
10 Petabytes is 10,000 TB. That's only 454 spinning disk drives (22 TB each). Approximate this as 600 with Raid 5. You can easily fit this in 100 desktops. There's tons of OSS to manage all layers of this stuff now.
You can have a few petabytes in a single rack.
50-ish drives in a single 4U server is done.
Re: How Canva saves Amazon S3 costs
#10Seems crazy to run such large infrastructure on a major cloud. Tens of millions wasted. Canva is likely trapped in S3 never to exit. The cost of getting their data out makes it impossible. S3…. the Hotel California of the cloud. You can check in any time you like but you can never leave. S3 is 9 cents per GB egress fees. Cloudflare R2 charges zero egress fees. At list price of 9 cents per GB it would cost Canva about…
Yup, we just moved large chunks of our SaaS platform from the cloud to dedicated servers with Kubernetes serving as an internal cloud. Ended up reducing our cost to 30% (70% saved!). We realized that instead of using scaling for peaks, by having additional dedicated servers available full time - it still worked out significantly cheaper. We also moved a lot of our internal processing to run in specific windows of tim…
Sorry what’s an “internal cloud”? Please tell me we haven’t renamed on-prem.