Live data from Hacker News

How Figma's databases team lived to tell the scale

figma.com

61–70 of 233 posts

Re: How Figma's databases team lived to tell the scale

#61

One thought that comes up: Wouldn’t it be easier to have each customer in their own (logical) database? I mean, you don’t need transactions across different customers, right? So you’re essentially solving a harder problem than the one you’ve got. Not sure postgres (logical) databases would scale that well, but don’t see a principal reason why it couldn’t. Has anyone explored this further?

Nile is a Serverless Postgres that virtualizes tenants/customers. It is specifically built for SaaS companies similar to Figma https://www.thenile.dev/. I am the CEO of Nile.

Re: How Figma's databases team lived to tell the scale

#62
post #38

Am I the only one finding the layout of this blog distracting? Kind of disappointing from a UX company. The images are also massive, the page was 42.21mb! Good article none the less! Always appreciate when companies like Figma document technical challenges.

It seems like someone at Figma decided to use the latest CSS tricks they just had discovered. Changing background color? Come on.

I thought this was a bug at first. Does anyone actually want this?

Re: How Figma's databases team lived to tell the scale

#64

I see they don't mention Citus ( https://github.com/citusdata/citus ), which is already a fairly mature native Postgres extension. From the details given in the article, it sounds like they just reimplemented it. I wonder if they were unaware of it or disregarded it for a reason —I currently am in a similar situation as the one described in the blog, trying to shard a massive Postgres DB.

I have worked on teams that have both sharded and partitioned PostgreSQL ourselves (somewhat like Figma) (Postgres 9.4-ish time frame) as well as those that have utilized Citus. I am a strong proponent of Citus and point colleagues in that direction frequently, but depending on how long ago Figma was considering this path I will say that there were some very interesting limitations to Citus not that long ago.

For example, it was only 2 years ago that Citus allowed the joining of data in "local" tables and data retrieved from distributed tables (https://www.citusdata.com/updates/v11-0). In this major update as well, Citus enabled _any_ node to handle queries, previously all queries (whether or not it was modifying data) had to go through the "coordinator" node in your cluster. This could turn into a pretty significant bottleneck which had ramifications for your cluster administration and choices made about how to shape your data (what goes into local tables, reference tables, or distributed tables).

Again, huge fan of Citus, but it's not a magic bullet that makes it so you no longer have to think about scale when using Postgres. It makes it _much_ easier and adds some killer features that push complexity down the stack such that it is _almost_ completely abstracted from application logic. But you still have be cognizant of it, sometimes even altering your data model to accommodate.

Re: How Figma's databases team lived to tell the scale

#65

> NoSQL databases are another common scalable-by-default solution that companies adopt as they grow. However, we have a very complex relational data model built on top of our current Postgres architecture and NoSQL APIs don’t offer this kind of versatility. As I understand it, NoSQL is for people who need a backend that ingests just about any unstructured data, for teams that may not have a complex relational model w…

What is your exact question? To me it makes sense that you’d not want to use NoSQL if you’re dealing with data that’s already relational, and heavily leveraging features common in relational DBs that may not come out of the box with NoSQL DBs.

They’re saying basically that NoSQL DBs solve a lot of horizontal scaling problems but aren’t a good fit for their highly relational data, is my understanding. Not that they can’t get NoSQL functionality at eg the query level in relational DBs.

Re: How Figma's databases team lived to tell the scale

#66
post #25

Earlier quoted context omitted.

Never a good idea to rely on Google proprietary tech (unless you are Google)... it could be sunset at any time without warning. I use GCP but I try my best to stay Google agnostic (avoid GCP-only offerings, etc) so that I can move to AWS if Google pulls the rug out from under me.

GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.

That’s what AppEngine customers thought.

Re: How Figma's databases team lived to tell the scale

#68

> NoSQL databases are another common scalable-by-default solution that companies adopt as they grow. However, we have a very complex relational data model built on top of our current Postgres architecture and NoSQL APIs don’t offer this kind of versatility. As I understand it, NoSQL is for people who need a backend that ingests just about any unstructured data, for teams that may not have a complex relational model w…

I think they're equating relation database with databases with ACID guarantees, as thats basically a full overlap on a venn diagram.

And we all know that acid has to go at some scale, even if that scale keeps getting pushed further out as our hardware gets better.

(Same with the relational guarantees that eat performance... But only once you've reached a certain amount of throughput and data)

Re: How Figma's databases team lived to tell the scale

#70

Earlier quoted context omitted.

GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.

That’s what AppEngine customers thought.

I worked on AppEngine (well, Serverless Compute) at Google for over 4 years and left on Friday. Did something happen to AppEngine in the last week?
Post reply on HN