71–80 of 113 posts
Is the hash based approach scalable? A list operation would require contacting every single shard which seems outrageously expensive. It seems like you want some locality but I’m not aware of locality-preserving hash functions (seems like a contradiction but maybe people here have encountered one?)
Is the hash based approach scalable? A list operation would require contacting every single shard which seems outrageously expensive. It seems like you want some locality but I’m not aware of locality-preserving hash functions (seems like a contradiction but maybe people here have encountered one?)
It works because most operations don't need to "list all of everything"
Also, I’m not saying “list all”. I’m saying even list 10 or list 1000 is the same problem - you still have to contact all servers in your cluster to do a map/reduce to get the result. Sure, list operations may be less common but their cost seems exponentially more expensive.
Earlier quoted context omitted.
> In a world where things like Dynamo/Cassandra or Spanner/Cockroach exist, manually-sharded DB solutions are pretty much entirely obsolete. Not really. Cassandra is a write optimised, slow, network and IO heavy beast thats a pain in the arse to administer. We replaced a 6 node m4.4xlarge with a single db.m4.2xlarge. on postgres. You need to pick your DB to match your data, not the otherway around.
Spanner and Cassandra really shouldn't be in the same sentence. They are optimized for very different use cases. The "obsolete" part of that quote does apply to Spanner, TiDB, and CockroachDB in my experience. I haven't used Yugabyte, but the other sharded databases, including Vitess (TiDB is what Vitess is trying to be), actually make life harder.
Not true. Vitess is far more proven.
> I suppose the more fundamental question is: why are you not using a database that does sharding for you? Over the past few years the so-called “serverless” database has gotten a lot more traction. Starting with the infamous Spanner paper, many have been thinking about how running a distributed system should be native to the database itself, the foremost of which has been CockroachDB. You can even run cloud Spanner…
btw, [distributed] Postgres iirc was never as stellar the single node (the stuff we sing praises of) vs the distributed deployment. I'm sure it has improved significantly.
> "manual operations required to reshard or rebalance from time to time. When it's part of the database itself, all those problems just... disappear."
Not really correct.
* "Choosing the right keys can help Spanner evenly distribute data and processing to avoid hotspots"
https://cloud.google.com/spanner/docs/schema-design
https://cloud.google.com/blog/topics/developers-practitioner...
** https://static.googleusercontent.com/media/research.google.c...
[Spanner certainly did -not- start off as a distributed RDBMS. Because that project would have never been given a green light. Because it is understood just how complex that system would need to be. It started off as a distributed k/v. That's it.]
"[I]n many ways today’s Spanner is very different from what was described [in original Spanner whitepaper]"
...
"The initial focus of Spanner was on scalability and fault-tolerance, much as was the case for many systems at Google. In the last few years Spanner has increasingly become a fully-fledged database system. Production deployment with internal customers taught us a great deal about the requirements of web scale database applications, and shaped the techniques we presented in this paper. Aggressive focus on horizontal scalability enabled widespread deployment without investing too heavily in single machine performance. The latter is one of the areas where we are making continuous improvements, such as upgrading lower level storage to Ressi."
"The original API of Spanner provided NoSQL methods for point lookups and range scans of individual and interleaved tables. While NoSQL methods provided a simple path to launching Spanner, and continue to be useful in simple retrieval scenarios, SQL has provided significant additional value in expressing more complex data access patterns and pushing computation to the data."
> I suppose the more fundamental question is: why are you not using a database that does sharding for you? Over the past few years the so-called “serverless” database has gotten a lot more traction. Starting with the infamous Spanner paper, many have been thinking about how running a distributed system should be native to the database itself, the foremost of which has been CockroachDB. You can even run cloud Spanner…
How does cockroach compares in terms of performance to manual sharded databases ? My intuition is that a properly sharded database will perform faster-or-same as a non-sharded one in all scenarios. Whereas automatically-sharded database will actually perform worst until you start reaching critical traffic that a single instance won't handle no matter what. Am i wrong ?
There is no intrinsic performance penalty for automatic sharding, it can be as fast or faster than any other scheme. The "faster" cases are when the sharding structure must be adaptive to have uniform distribution of data and load, such as when the key distribution is intrinsically unpredictable or when you are doing multi-attribute sharding. In these cases, effective sharding can only be created by software at runtime.
Earlier quoted context omitted.
> PS Azure Cosmos is not a real product, it is a beta toy that Microsoft just has made expensive enough that people think it cannot possibly be as bad as it is.. what kind of problems you ran into with Cosmos?
It’s supposed to be MongoDB compatible (if I’m not mistaken). Last I checked, that “compatibility” run at least a couple major version behind the real MongoDB. To be fair, the same is probably also true with AWS’s DocumentDB
Earlier quoted context omitted.
Spanner and Cassandra really shouldn't be in the same sentence. They are optimized for very different use cases. The "obsolete" part of that quote does apply to Spanner, TiDB, and CockroachDB in my experience. I haven't used Yugabyte, but the other sharded databases, including Vitess (TiDB is what Vitess is trying to be), actually make life harder.
> (TiDB is what Vitess is trying to be) Not true. Vitess is far more proven.
At my current job, we have reached a scale and use case that requires either manually sharding the database or using a distributed one. 20 TB including indexes, 10-20 tables with >1B rows, and spikes of up to 10K/s transactions. We put Vitess, TiDB, CockroachDB, and Spanner head-to-head, including running production scale load tests. Spanner won out because it is far superior to them all for scaling geo-replication and sysadmin. But TiDB was a close second because it just works, scales, and fast. Vitess on the other hand was extremely buggy and has a very long, undocumented list of unsupported SQL features. Yes Vitess has better press and more well known, but it is an inferior technology imo. TiDB is already what Vitess claims to be.
And the best sharding? Doing multi-tenant. Sharding at table level is very complex and expensive. Fully give a single DB per tenant is very practical, and the reasons to do sharding (like reporting) is where you do the other fancy things (like ship events in to kafka, etc, etc). Also, most issues of scalability are dominated by a few tenants that consume most resources, and distribute the loads is more easy per-tenan…
> I suppose the more fundamental question is: why are you not using a database that does sharding for you? Over the past few years the so-called “serverless” database has gotten a lot more traction. Starting with the infamous Spanner paper, many have been thinking about how running a distributed system should be native to the database itself, the foremost of which has been CockroachDB. You can even run cloud Spanner…
How does cockroach compares in terms of performance to manual sharded databases ? My intuition is that a properly sharded database will perform faster-or-same as a non-sharded one in all scenarios. Whereas automatically-sharded database will actually perform worst until you start reaching critical traffic that a single instance won't handle no matter what. Am i wrong ?
This is workload dependent, if your queries only go to one shard, this is likely true. But if you have cross shard queries, then it’s no longer true.
A favorite resource: https://learn.microsoft.com/en-us/azure/architecture/pattern... Microsofts Azure Cloud Patterns is some of the best documentation out there. It's not Azure centric. It focuses on why you may want to do something and describes the techniques that are commonly used to solve it. It's pretty great.