How Figma's databases team lived to tell the scale • Hacker News Board

Re: How Figma's databases team lived to tell the scale

#231

Post by imbradn » Mon, Mar 18, 2024, 11:31 PM UTC

thekhatribharat wrote: post #146

1. They mention that the largest tables ran into several TBs, and they would have soon topped the max IOPS supported by RDS. RDS for PostgreSQL peaks at 256,000 IOPS for a 64 TB volume. For a multi-AZ setup, this costs ~$70K/mo. 2. Let's assume the final outcome was a 5-way shard with each shard supporting ~50,000 IOPS and ~12 TB data. For a multi-AZ setup, this costs ~$100K/mo. 3. It took 9 months to shard their fir…

IOPS isn’t a linear thing here. The vacuums needed to prevent transaction wraparound (vaccum Freeze)can’t be throttled and are much more expensive than regular vacuums. By splitting the tables they are likely reducing the need for those vacuums (by a large margin) and significantly reducing IOPS needs.

Re: How Figma's databases team lived to tell the scale

#232

Post by imbradn » Mon, Mar 18, 2024, 11:36 PM UTC

jamra wrote: post #165

kamikaz1k wrote: post #163

Earlier quoted context omitted.

Never used cockroach so pardon my ignorance, but are there no operational challenges with running/using them? Or are they the same challenges? And how compatible is it from an application developer perspective?

CRDB is Postgres compliant so the wire protocol and SQL syntax is all Postgres. It should be a 1 to 1.

Except for the un-implemented features which they might need.

It also uses serializable isolation and in their implementation reads are blocked by writes unlike in Postgres. Those are both significant changes that can have far reaching application impacts

Re: How Figma's databases team lived to tell the scale

#233

Post by rockostrich » Thu, Mar 21, 2024, 6:49 PM UTC

marcinzm wrote: post #131

rockostrich wrote: post #54

Earlier quoted context omitted.

GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.

AI Platform is a recent example that’s been deprecated.

Wasn't it just superseded by Vertex AI? According to the banner announcement in the docs: > All the functionality of legacy AI Platform and new features are available on the Vertex AI platform.