Why Has Figma Reinvented the Wheel with PostgreSQL?
1–10 of 98 posts
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#2Nice article, I also wondered why they omitted citus. Is there any plan from rds to offer it?
Obviously it could interfere with demand for aurora limitless
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#3Good points. Although, having worked on many high scale architectures before, I always err on the side of thinking that any technical solution of this magnitude would have far too many nuances for a blog post to capture. And I believe Figma’s post also talks mainly about the common denominator that’s easier to communicate to the external world.
For me to understand their choices, I’ll first have to understand their system, which can be a task in itself. Without that, I’d not become aware of the nuances, only general patterns.
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#4The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there.
I have seen this several times before and it's always a symptom of having too many engineers working below the waterline. Rather than work on the actual customer-facing problem, let's port the backend to do event-sourcing/cqrs, move all our infrastructure to k8s, change language from x to y etc.
These are all what I would call "internal goals" (ie they may or may not be necessary or even essential to progress but are not directly customer-visible in their outcomes even if they may enable customer features to be built or indirectly improve the customer experience later) and need to be held to an extremely high level of scrutiny.
If you're amazon/google/meta and you need to do this because of extreme user scale I might believe you. If you're CERN or someone and you need to do this because of absolutely ridiculous data scale I might believe you. The idea that it's better for figma to write their own sharding solution than it is to port to one of the existing ones just doesn't pass even the most basic sniff test.
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#5The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#6The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?
I just read through several thousand lines of code re-implementing the concept of a distributed queue from the ground up... for an application that has maybe a few hundred users. And doesn't need queues, at all.
This issue is so pervasive that we've all just assumed that it must be necessary.
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#7Now seems a good time to point out, the wheel has literally been reinvented over and over again. The wheels of yesterday were terrible. Each version gets better. It's fine, reinvent away folks :)
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#8The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?
Absolutely not. Citus would have solved this problem. Or move to MySQL and use PlanetScale etc.
Second best option is ability to easily create prod environments, and then give those to your biggest customers (bigname.figma.com) etc. No single figma customer will go beyond an i3.metal for the DB, or the app.
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#9Earlier quoted context omitted.
I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?
Absolutely not. Citus would have solved this problem. Or move to MySQL and use PlanetScale etc. Second best option is ability to easily create prod environments, and then give those to your biggest customers (bigname.figma.com) etc. No single figma customer will go beyond an i3.metal for the DB, or the app.
So I just read the article - they were on RDS so Citus wasn't an option.
They also stated it was too risky to migrate data stores on the timeline they were working within
Those all seem like measured engineering decisions AFAICT
Re: Why Has Figma Reinvented the Wheel with PostgreSQL?
#10Earlier quoted context omitted.
I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?
I just read through several thousand lines of code re-implementing the concept of a distributed queue from the ground up... for an application that has maybe a few hundred users. And doesn't need queues, at all. This issue is so pervasive that we've all just assumed that it must be necessary.
I couldn't get the context of this response - is this application you read unrelated to the featured article?
I just read the article and from what I can tell the Figma team made a somewhat reasonable sounding decision