21–30 of 98 posts
Also these can add unnecessary complexity by having features you don't need. Or they might be missing features you need. Contributing up-stream can be difficult and there might be conflicts of interest especially for projects which have separate paid version.
The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
That said, I've seen plenty of unprofitable startups with high burn rate play this game. That seems a bit suicidal to me.
The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
The article suggests a different reason. What would be your approach if you wanted to stay on RDS? > So, now, let me speculate. The real reason why Figma reinvented the wheel by creating their own custom solution for sharding might be as straightforward as this — Figma wanted to stay on RDS, and since Amazon had decided not to support the CitusData extension in the past, the Figma team had no choice but to develop th…
The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
I suppose Figma might just be beyond the "let's find the fastest/cheapest way to get this working" point. I believe it makes sense for a company in that stage to mess about a bit, find different (maaaybe even better) ways of doing things, keep the engineering work interesting to attract/retain talent, be OK with the inevitable waste involved in that game. If you're chasing the global maximum, you shouldn't get too ob…
The article implies otherwise. E.g. it quotes Figma saying: “Given our very aggressive growth rate, we had only months of runway remaining.”
The answer is obvious: they invented their own sharding solution because it's a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there. I have seen this several times before and it's always a symptom of having too many…
Also, I’m assuming Amazon or Google will sometimes roll their own solutions on problems of a scale in the same ballpark as Figma’s.
But anyhow, what’s the scale at which this becomes acceptable, exactly? Is there a magical number which serves as a universal threshold? Or is there - like in all engineering decisions - a very concrete economic case for which you and I both lack a lot of the requisite context and inputs?
Earlier quoted context omitted.
I suppose Figma might just be beyond the "let's find the fastest/cheapest way to get this working" point. I believe it makes sense for a company in that stage to mess about a bit, find different (maaaybe even better) ways of doing things, keep the engineering work interesting to attract/retain talent, be OK with the inevitable waste involved in that game. If you're chasing the global maximum, you shouldn't get too ob…
> I suppose Figma might just be beyond the "let's find the fastest/cheapest way to get this working" point. The article implies otherwise. E.g. it quotes Figma saying: “Given our very aggressive growth rate, we had only months of runway remaining.”
Aren't there any good managed postgres solutions supporting citus? The decision here seems to have been to invent a whole new sharding solution instead of building enough in house DBA to self-host postgres (if you want to stay on Amazon, you can use any extension you want on EC2). Speaks for the state of engineering right now.
I'm at a company that is weighing a very similar decision (we are on RDS Postgres with a rapidly growing database that will require some horizontal partitioning). There really isn't an easy solution. We spoke to people who have done sharding in-house (Figma, Robinhood) as well as others who migrated to natively distributed systems like Cockroach (Doordash). If you decide to move off of RDS but stay on Postgres, you c…