Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Figma's databases team lived to tell the scale

figma.com

Post reply on HN
Print view View original

41–50 of 233 posts

Re: How Figma's databases team lived to tell the scale

#41

Post by junto »

adastral wrote: post #31

I see they don't mention Citus ( https://github.com/citusdata/citus ), which is already a fairly mature native Postgres extension. From the details given in the article, it sounds like they just reimplemented it. I wonder if they were unaware of it or disregarded it for a reason —I currently am in a similar situation as the one described in the blog, trying to shard a massive Postgres DB.

Before clicking on the article I assumed it was Citus, and was surprised when it wasn’t.

Maybe because CitusData was bought by Microsoft around the same time, so Microsoft could create “Azure Cosmos DB for Postgres Cluster”, yet another one of Microsoft’s typical product naming crapshoots.

Re: How Figma's databases team lived to tell the scale

#42

Post by sgarland »

adastral wrote: post #31

I see they don't mention Citus ( https://github.com/citusdata/citus ), which is already a fairly mature native Postgres extension. From the details given in the article, it sounds like they just reimplemented it. I wonder if they were unaware of it or disregarded it for a reason —I currently am in a similar situation as the one described in the blog, trying to shard a massive Postgres DB.

I thought of that as well. The only thing I could think of is that they mentioned that they don't want to move off of RDS, and there is 0% chance of Citus coming to AWS since Microsoft bought them.

Re: How Figma's databases team lived to tell the scale

#44

Post by willsmith72 »

eatonphil wrote: post #32
bjornsing wrote: post #29

One thought that comes up: Wouldn’t it be easier to have each customer in their own (logical) database? I mean, you don’t need transactions across different customers, right? So you’re essentially solving a harder problem than the one you’ve got. Not sure postgres (logical) databases would scale that well, but don’t see a principal reason why it couldn’t. Has anyone explored this further?

I imagine it only gets you so far. What do you do about customers like Walmart or Oracle? Hundreds, if not thousands, of users all leaving hundreds of comments on Figma files every day. If you want good latency without giving up strong consistency (which the article says they want) you'll need to keep sharding.

A single db can handle that load easily

Re: How Figma's databases team lived to tell the scale

#45

Post by bearjaws »

vinner_roy wrote: post #43

Could you use Aurora Limitless for this instead? https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-au...

I doubt even VC money can afford this service.

Serverless Aurora is incredibly expensive for most workloads. I have yet to find a use case for any SaaS product that is used >4 hours a day. Since all my products span at least 3 time zones there is at least 12 hours of activity a day.

Re: How Figma's databases team lived to tell the scale

#46

Post by aeyes »

bjornsing wrote: post #29

One thought that comes up: Wouldn’t it be easier to have each customer in their own (logical) database? I mean, you don’t need transactions across different customers, right? So you’re essentially solving a harder problem than the one you’ve got. Not sure postgres (logical) databases would scale that well, but don’t see a principal reason why it couldn’t. Has anyone explored this further?

I have pondered about this for quite some time and came to the conclusion that it would make schema migrations more difficult to handle. I think Shopify is using an approach which is similar to what you are describing. The advantage is that you don't end up with hot shards because you can move around large customers independently.

In practice there isn't a big difference, they just colocate several customers according to their sharding key in the same logical database.

Re: How Figma's databases team lived to tell the scale

#47

Post by groestl »

szundi wrote: post #20
hipadev23 wrote: post #15

Earlier quoted context omitted.

The problem is nobody outside Google trusts them to run or operate anything. Edit: To the Googlers downvoting these comments. Your behavior only reinforces our views.

Google means: good chance discontinued after you have worked out the bugs and have a stable system at last

This is definitely not the case with their core cloud products.

> almost every project not using Spanner was moving to Spanner

This even includes Datastore. Even Datastore moved to Spanner.

Re: How Figma's databases team lived to tell the scale

#48

Post by renegade-otter »

TkTech wrote: post #38

Am I the only one finding the layout of this blog distracting? Kind of disappointing from a UX company. The images are also massive, the page was 42.21mb! Good article none the less! Always appreciate when companies like Figma document technical challenges.

It seems like someone at Figma decided to use the latest CSS tricks they just had discovered. Changing background color? Come on.
Post reply on HN