91–100 of 233 posts
Am I the only one finding the layout of this blog distracting? Kind of disappointing from a UX company. The images are also massive, the page was 42.21mb! Good article none the less! Always appreciate when companies like Figma document technical challenges.
Earlier quoted context omitted.
how "massive" is massive in your case?
I've had CitusDB running across 68 bare metal machines (40 vCPU, 768GiB ram, 20TiB of storage each + 40GiB network links) and it ran decently well. Not sure what your definition of massive is, I think Spanner would easily beat it. Also, it's very use-case dependent, you can't "just use" Citus for everything, it's not quite as flexible as a bog-standard pgsql install due to the way it's sharding, you have to be a tad…
How transactions work when you end up querying different shards?
One thought that comes up: Wouldn’t it be easier to have each customer in their own (logical) database? I mean, you don’t need transactions across different customers, right? So you’re essentially solving a harder problem than the one you’ve got. Not sure postgres (logical) databases would scale that well, but don’t see a principal reason why it couldn’t. Has anyone explored this further?
Actually, Apple does this for iCloud! They use FoundationDB[1] to store billions of databases, one for each user (plus shared or global databases). See: https://read.engineerscodex.com/p/how-apple-built-icloud-to-... Discussed on HN at the time: https://news.ycombinator.com/item?id=39028672 [1]: https://github.com/apple/foundationdb https://en.wikipedia.org/wiki/FoundationDB
This is sort of true and sort of false. When you think of a "database", if you're thinking of a Postgres database, you're way off the reality of what "database" means here.
FoundationDB has a concept called "layers", and essentially they have created a layer that looks like a separate database on top of a layer that is separately encrypted groups of keys. They don't have billions of FoundationDB clusters or machines, and at the infra level, i.e. the instances of the FoundationDB server software, it's unaware of individual "databases".
A closer analogy would be like having billions of tables, but even that isn't accurate because in relational databases a table is usually a notably more static concept than data in a table. The closest analogy would be that each of the billions of users has a bunch of rows with a user-id field on them, and there's a proxy that filters everything such that you can view the table as if it only had one user's data in it.
To be clear, FoundationDB is awesome and Apple have done some really cool stuff with it, but it's less crazy/impressive than it sounds.
Earlier quoted context omitted.
Before clicking on the article I assumed it was Citus, and was surprised when it wasn’t. Maybe because CitusData was bought by Microsoft around the same time, so Microsoft could create “Azure Cosmos DB for Postgres Cluster”, yet another one of Microsoft’s typical product naming crapshoots.
> yet another one of Microsoft’s typical product naming crapshoots. Well said. I haven't seen any company as terrible as Microsoft at naming things. Anyone know why?
Earlier quoted context omitted.
Can you offer insight into what a better approach might have been?
As others have mentioned, moving to per tenant databases can really simplify things at scale and doesn't leave a massive amount of engineering complexity and debt in its wake. I feel sorry for the team managing this 5 years from now.
Am I the only one finding the layout of this blog distracting? Kind of disappointing from a UX company. The images are also massive, the page was 42.21mb! Good article none the less! Always appreciate when companies like Figma document technical challenges.
It seems like someone at Figma decided to use the latest CSS tricks they just had discovered. Changing background color? Come on.
Coming from Google, where Spanner is this magical technology that supports infinite horizontal sharding with transactions and has become the standard storage engine for everything at Google (almost every project not using Spanner was moving to Spanner), I'm curious how Figma evaluated Cloud Spanner. Cloud Spanner does have a postgres translation layer, though I don't know how well it works. It seems like they've (hop…
Why would anyone marry themselves to Google? That sounds like the most boneheaded move possible. First, you should never be beholden to a single vendor for the most critical technological underpinnings. You're backing yourself into a corner. But more importantly, Google can't even figure out how to prioritize their cloud efforts. That's not a good partnership to be in for anyone except Google. I wouldn't care if my s…
Earlier quoted context omitted.
Before clicking on the article I assumed it was Citus, and was surprised when it wasn’t. Maybe because CitusData was bought by Microsoft around the same time, so Microsoft could create “Azure Cosmos DB for Postgres Cluster”, yet another one of Microsoft’s typical product naming crapshoots.
> yet another one of Microsoft’s typical product naming crapshoots. Well said. I haven't seen any company as terrible as Microsoft at naming things. Anyone know why?
At a previous employer, I saw several cool-ish open source projects instantly doomed to obscurity by picking a name that either completely duplicated the name of an existing OSS project or were guaranteed to have terrible SEO for another reason.
However, Microsoft seems to have a unique crossover of fragmented business units and centralized marketing. That's why you end up with Azure -> Subproject -> Actual Product/Service word soup. Perviously, they did this with the Windows Live brand from 2005-2012, and "Xbox" for a wide range of gaming projects (many of which were on PC).