Can I install this locally? Will it work without an internet connection? Is it fully open source? From what I can tell, the answer to all three questions is no. Is it yet another example of vendor lock-in? Yes.
If it is MySQL compatible it's not vendor lock in.
It's more like a clever trap, first hit is free etc.
Will be interesting to follow this. I'm not db savvy so I don't mind leaving that job to someone else.
This is wonderful. I've been developing with relational databases for 12 years and couldn't be more excited about this. Scaling down to 0 opens up a world of opportunities when it comes to team development workflows. The idea of quick environments per pull request is within reach. Not to mention that it scales with you. The one thing I'm curious about is how it compares with CockroachDB
Quick environment per pull request is available with SQLite now :)
Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.
As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much better scaling. The big limitation on Vitess is cross-shard transactions, and the fact you have to make sure your schema has a clear cut sharding key (like your tenant ID) that works nicely with your application needs. The alternative for scaling transactional SQL DBs in a multi-master fashion are the "NewSQL" DBs like yugabyte and cockroachdb which are basically document DBs with a partially implemented postgres frontend, so don't have the full feature set of your SQL engine like Vitesse does but don't require so much attention to sharding. These are oversimplifications of the actual mechanisms, but give a basic overview of the tradeoffs involved, please feel free to correct me on any inaccuracies as I'm not an expert in DBs.
Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.
As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much be…
I was Chief Architect at Slack from 2016 to 2020, and was privileged to work with the engineers who were doing the work of migrating to Vitess in that timeframe.
The assumption that tenants are perfectly isolated is actually the original sin of early Slack infrastructure that we adopted Vitess to migrate away from. From some earlier features in the Enterprise product (which joins lots of "little Slacks" into a corporate-wide entity) to more post-modern features like Slack Connect (https://slack.com/help/articles/1500001422062-Start-a-direct...) or Network Shared Channels (https://slack.com/blog/news/shared-channels-growth-innovatio...), the idea that each tenant is fully isolated was increasingly false.
Vitess is a meta-layer on top of MySQL shards that asks, per table, which key to shard on. It then uses that information to maintain some distributed indexes of its own, and to plan the occasional scatter/gather query appropriately. In practice, simply migrating code from our application-sharded, per-tenant old way into the differently-sharded Vitess storage system was not a simple matter of pointing to a new database; we had to change data access patterns to avoid large fan-out reads and writes. The team did a great write-up about it here: https://slack.engineering/scaling-datastores-at-slack-with-v...
Can I install this locally? Will it work without an internet connection? Is it fully open source? From what I can tell, the answer to all three questions is no. Is it yet another example of vendor lock-in? Yes.
If it is MySQL compatible it's not vendor lock in. It's more like a clever trap, first hit is free etc. Will be interesting to follow this. I'm not db savvy so I don't mind leaving that job to someone else.
Vender lock-in isn't just about data formats, it's about the overall makeup of your infrastructure and development processes. If an org builds around the particular way PlanetScale manages DB branches and other minutiae of their service, they can't simply replace that with an in-house DB server overnight.
As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much be…
I was Chief Architect at Slack from 2016 to 2020, and was privileged to work with the engineers who were doing the work of migrating to Vitess in that timeframe. The assumption that tenants are perfectly isolated is actually the original sin of early Slack infrastructure that we adopted Vitess to migrate away from. From some earlier features in the Enterprise product (which joins lots of "little Slacks" into a corpor…
Definitely wasn't expecting the chief architect at Slack to reply to that example, really appreciate the response, HN is such a blessing in that regard :). The scaling datastores at slack is a super interesting read aswell thanks, does make me wonder if there was a fully 100% MySQL compatible version of yugabyte/spanner etc if that would have shifted the decision.