Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Figma's databases team lived to tell the scale

figma.com

Post reply on HN
Print view View original

101–110 of 233 posts

Re: How Figma's databases team lived to tell the scale

#101

Post by infecto »

eatonphil wrote: post #32
bjornsing wrote: post #29

One thought that comes up: Wouldn’t it be easier to have each customer in their own (logical) database? I mean, you don’t need transactions across different customers, right? So you’re essentially solving a harder problem than the one you’ve got. Not sure postgres (logical) databases would scale that well, but don’t see a principal reason why it couldn’t. Has anyone explored this further?

I imagine it only gets you so far. What do you do about customers like Walmart or Oracle? Hundreds, if not thousands, of users all leaving hundreds of comments on Figma files every day. If you want good latency without giving up strong consistency (which the article says they want) you'll need to keep sharding.

I bet it gets you further than you imagine. Entirely depends on the backend services and what they touch but in this scenario you would be deploying/scaling that service based on the customer seat size. I suspect that even for large enterprise customers, the users actively touching Figma are not reaching he thousands but I am happy to be wrong.

After all, Stackoverflow is running off of a handful of machines.

Re: How Figma's databases team lived to tell the scale

#102

Post by wayne-li2 »

gen220 wrote: post #85
gshulegaard wrote: post #64

Earlier quoted context omitted.

I have worked on teams that have both sharded and partitioned PostgreSQL ourselves (somewhat like Figma) (Postgres 9.4-ish time frame) as well as those that have utilized Citus. I am a strong proponent of Citus and point colleagues in that direction frequently, but depending on how long ago Figma was considering this path I will say that there were some very interesting limitations to Citus not that long ago. For exa…

You also benefit from the tailwind of the CitusData team making continued improvement to the extension, whereas an in-house system depends on your company's ability to hire and retain people to maintain + improve the in-house system. It's hard to account for the value of benefits that have yet to accrue, but this kind of analysis, even if you pretty heavily-discount that future value, tilts the ROI in favor of soluti…

Huh, I would have thought the opposite. Companies at Figma size are easily able to hire talent to maintain a core part of their engineering stack. On the other hand, they retain no control of Citus decision making. Those tailwinds could easily have been headwinds if they went in a direction that did not suit Figma.

Re: How Figma's databases team lived to tell the scale

#103

Post by zenbowman »

nosefrog wrote: post #9

Coming from Google, where Spanner is this magical technology that supports infinite horizontal sharding with transactions and has become the standard storage engine for everything at Google (almost every project not using Spanner was moving to Spanner), I'm curious how Figma evaluated Cloud Spanner. Cloud Spanner does have a postgres translation layer, though I don't know how well it works. It seems like they've (hop…

I wouldn't say "infinite", its still susceptible to read hotspotting; and while fine-grained locking enables generally higher write throughputs, you can still get in a situation where interconnected updates end up being pretty slow.

That said, its way better than anything else I've used in my career.

Re: How Figma's databases team lived to tell the scale

#104

Post by jerrygenser »

HermitX wrote: post #69

I This is an intriguing article, clearly showing the team's fondness for Postgres. However, Postgres is an OLTP product. I'm curious about what system Figma's data team uses for their data analysis tasks.

They mentioned analyzing query logs in Snowflake in this very article. So... at least they use Snowflake for some things?

Re: How Figma's databases team lived to tell the scale

#105

Post by jerrygenser »

rockostrich wrote: post #54
umvi wrote: post #25

Earlier quoted context omitted.

Never a good idea to rely on Google proprietary tech (unless you are Google)... it could be sunset at any time without warning. I use GCP but I try my best to stay Google agnostic (avoid GCP-only offerings, etc) so that I can move to AWS if Google pulls the rug out from under me.

GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.

IoT is one example of a big backbone service that was sunset.

Re: How Figma's databases team lived to tell the scale

#106

Post by jerrygenser »

weitendorf wrote: post #75
umvi wrote: post #25

Earlier quoted context omitted.

Never a good idea to rely on Google proprietary tech (unless you are Google)... it could be sunset at any time without warning. I use GCP but I try my best to stay Google agnostic (avoid GCP-only offerings, etc) so that I can move to AWS if Google pulls the rug out from under me.

I'm biased having worked on GCP, but I think GCP actually has a very good track record of not sunsetting entire products or removing core functionality. When I worked on AppEngine, I would often see apps written 10+ years ago still chugging along. It is true though that GCP sometimes sunsets specific product functionality, requiring changes on the customers' part. Some of these are unavoidable (eg committing to apply…

>It is true though that GCP sometimes sunsets specific product functionality, requiring changes on the customers' part. Some of these are unavoidable (eg committing to apply security patches to Python 2.7 given that the rest of the world is mostly not upstreaming these patches anymore), but not all of them.

A good example is probably IoT. I've heard first hand anecdotes of very difficult migrations off this service.

Re: How Figma's databases team lived to tell the scale

#107

Post by ghostly_s »

TkTech wrote: post #38

Am I the only one finding the layout of this blog distracting? Kind of disappointing from a UX company. The images are also massive, the page was 42.21mb! Good article none the less! Always appreciate when companies like Figma document technical challenges.

Odd, it looks perfectly bog-standard for me on Firefox and iOS, aside from the lavender bg which I quite like. I even disabled my ad-blocker to see if it made a difference, no changing bg colors for me...

Re: How Figma's databases team lived to tell the scale

#108

Post by simonw »

skunkworker wrote: post #92
dijit wrote: post #39

Earlier quoted context omitted.

I've had CitusDB running across 68 bare metal machines (40 vCPU, 768GiB ram, 20TiB of storage each + 40GiB network links) and it ran decently well. Not sure what your definition of massive is, I think Spanner would easily beat it. Also, it's very use-case dependent, you can't "just use" Citus for everything, it's not quite as flexible as a bog-standard pgsql install due to the way it's sharding, you have to be a tad…

What is your definition of "decently well", and is your primary cluster (without replicas) above 1PB?

They said 20TiB * 68, which I think is 1.5PB.

Re: How Figma's databases team lived to tell the scale

#109

Post by simonw »

dakiol wrote: post #28

I cannot help but think this all sounds pretty much like a hack (a clever one, though). We do not handle, let's say low-level I/O buffering/caching, by ourselves anymore, right? (at least the folks doing web development/saas). We rely instead of the OS APIs, and that's good. I think we are missing something similar but for db sharding. It seems to me that we are still missing some fundamental piece of technology/infr…

Citus and Vitess are examples of horzontal data sharding technology for PostgreSQL and MySQL respectively.

At Figma's size there are sometimes reasons to roll your own, which I think they explain pretty clearly in the article. They wanted a solution they could incrementally engineer onto their existing stack without doing a full rewrite or lift-and-shift to something else.

Re: How Figma's databases team lived to tell the scale

#110

Post by simonw »

TheP1000 wrote: post #89
dwaltrip wrote: post #84

Earlier quoted context omitted.

Can you offer insight into what a better approach might have been?

As others have mentioned, moving to per tenant databases can really simplify things at scale and doesn't leave a massive amount of engineering complexity and debt in its wake. I feel sorry for the team managing this 5 years from now.

Moving to a per-tenant database sounds like even more work to me than moving to shards. Moving to per-tenant means rewriting _everything_ - moving to shards has you rewriting a lot less.
Post reply on HN