Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Canva saves Amazon S3 costs

canva.dev

Post reply on HN
Print view View original

161–170 of 181 posts

Re: How Canva saves Amazon S3 costs

#161

Post by kazen44 »

betaby wrote: post #101
shadowtree wrote: post #96

Earlier quoted context omitted.

Most businesses treat cloud like a utility, not tech. Don't want to generate their own power, deal with sewage ... and compute. Which in 99.9% of the cases is the right decision. Sorry for all you infra techies, but that's the way tech matures.

Infra techies are fine. Likewise 'grid is mature' and yet most of buildings with floor space over NN m2 have team of electricians, being them direct hire or contractors. Same for the 'cloud' - 90%+ of the world deployed 'compute' is not cloud.

Also, infrastructure fundementals do not change, now matter where they are running.

As long as dev's don't seem to understand real world limitations, (for instance, a network cannot be instantanious, and latency is not consistent) and thus leave tons of performance on the table, i think the "infra guys" will be just fine.

Also, don't forget there is an entire world out there of people making sure connections to the cloud are even possible across the internet.

Re: How Canva saves Amazon S3 costs

#162

Post by kazen44 »

jabart wrote: post #134
littlestymaar wrote: post #132

Earlier quoted context omitted.

This is a marketing pitch for a cloud storage company and it shows: > - System hardware: $500,000 For one PB? Gold plated HDDs and cases I guess… > Assume 15% annual system maintenance over five years Do not run your hard drive cluster on the same floor as vibration-inducing machines, folks. /s According to Backblaze, even a 8 year-old hard drive is still twice more reliable than this, and 15% failure rate is what to…

Can you fit 10pb in a rack, sure if you want it all flash. You also need double that for mirroring, or quadruple that for raid 10. Double that if you want a mirrored copy of that array too. If you don't have high performance needs you can get by with slower hard drives. You also have the 25gb,40gb, or 100gb top of rack switch. Well you want two of those for redundancy. It starts to quickly add up. Annual system maint…

> Oct 2020, StorONE announced a 1PB All-Flash array for $499k.

mind you the price of flash storage has dropped significantly since 2020.

Hard disks are still being used, but having all flash systems is not uncommon.

Re: How Canva saves Amazon S3 costs

#163

Post by kazen44 »

icedchai wrote: post #109
colechristensen wrote: post #9

Earlier quoted context omitted.

You can have a few petabytes in a single rack. 50-ish drives in a single 4U server is done.

What about single points of failure? Replication? High availability? Disaster recovery? Backup / snapshots/ versioning? This is one of the benefits of the cloud.

half of these are already managed by something like ZFS.. (the only one i can think of which is not inherent in the Filesystem is High availability). But one should build that kind of redundancy on a higher level anyways. (multiple storage pools, replicating writes etc).

or, if you go fibre channel, just replicate the writes across multiple storage systems and be done with it. (this requires a fibre channel network though, which is neither easy nor cheap).

Re: How Canva saves Amazon S3 costs

#164

Post by colechristensen »

icedchai wrote: post #159
colechristensen wrote: post #152

Earlier quoted context omitted.

Plenty of tools for that on premises. Are you actually asking?

No, I'm just mentioning these things to show that the decision is more complex than a "4U server" vs "S3." S3/AWS gives you a ton of stuff you'd have to implement yourself. Plus you need to hire more people to manage it.

Minio does most of S3 for example, duh you would have to do some stuff yourself but you wouldn’t be implementing everything yourself from scratch, far from it.

https://min.io/

Re: How Canva saves Amazon S3 costs

#165

Post by Quarrel »

charcircuit wrote: post #140
Quarrel wrote: post #99

Earlier quoted context omitted.

Well, step 1, have petabytes of data in S3, then don't pay the sticker price. Step 2, write articles like this as part of your newly contracted "engagement" with Amazon to help justify the lower price.

>step 1, have petabytes of data in S3, then don't pay the sticker price. So pay Amazon a bunch for the privilege to have lower prices. How about just starting off somewhere where you don't have to commit to paying a lot of money to get a reasonable price?

Because when you start out you don't know you'll need petabytes of storage.

Canva grew very quickly. I don't think it was a terrible call to use the cloud at the time.

Re: How Canva saves Amazon S3 costs

#166

Post by electroly »

motoxpro wrote: post #42

A lot of people in this thread should start consultancies that transition companies off the cloud. Apparently, it would be easy for them to save tens of millions, in turn making millions for the consultant.

There is a lot of good money to be made for people who understand cloud cost structures and the alternative options available. You'll want to target mid-size businesses that bet on a cloud and were actually successful but are now drowning in cloud bills. Large companies will hire in-house expertise, and small companies don't spend enough for it to be worth hiring a contractor. But I really think there's a market of mid-size companies who don't know how to reduce their AWS bills. Cloud cost optimization (and perhaps migrations between clouds or to hybrid cloud/on-prem at times) would be a great business for a solo contractor. Being able to pay for your own contracting fee out of the savings is almost a given.

Re: How Canva saves Amazon S3 costs

#167

Post by shadowtree »

mschuster91 wrote: post #130
shadowtree wrote: post #111

Earlier quoted context omitted.

Work for a 30bn market cap company, all our stuff is built on AWS. Offices all around the globe. Not a single electrician on payroll. YMMV.

> Not a single electrician on payroll. Directly, yes maybe. Many companies don't have in-house experts (and get shafted by vendors as a result, as they're lacking the competence required to assess the quality of work). But indirectly, they all have. In Germany, at least, you're required to have electrical appliances inspected by a licensed professional (an electrician or otherwise qualified person) at least every two…

Utility is just an old term for "as a service".

Electricity as a Service, which includes payments for expert maintenance.

Thread is about AWS and merits of in-sourcing, which is not how the world operates.

Anyways.

Re: How Canva saves Amazon S3 costs

#168

Post by yencabulator »

shawabawa3 wrote: post #129
pc86 wrote: post #74

Earlier quoted context omitted.

I know we're just arguing about amounts at this point but that's pushing $2M a week . It seems on on-prem would make financial sense long before that.

Depends on your requirements. If you need 11 nines durability S3 is probably still cheaper

There's also the hybrids. Only storing "unlikely to be accessed" data on S3 would likely remove a huge part of the AWS bill, while significantly limiting the size of the on-prem data set.

Of all the possible related problems, object storage is the easiest.

Re: How Canva saves Amazon S3 costs

#169

Post by charcircuit »

Quarrel wrote: post #165
charcircuit wrote: post #140

Earlier quoted context omitted.

>step 1, have petabytes of data in S3, then don't pay the sticker price. So pay Amazon a bunch for the privilege to have lower prices. How about just starting off somewhere where you don't have to commit to paying a lot of money to get a reasonable price?

Because when you start out you don't know you'll need petabytes of storage. Canva grew very quickly. I don't think it was a terrible call to use the cloud at the time.

Cloud isn't the issue. Using overpriced services is the issue. Regardless of size they shouldn't have used S3.
Post reply on HN