Live data from Hacker News

PlanetScale – Database for Developers

planetscale.com

61–70 of 135 posts

Re: PlanetScale – Database for Developers

#61

Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.

IME the answer to "how did they make [hard to scale thing] easily scalable?" is usually that they introduce limitations in how you can use [hard to scale thing] so you can't use it in ways that are hard to scale, then automate scaling it in well-known ways for use cases that are so-limited. Vitesse's site mentions that it relies on horizontal sharding, so right off the bat, my guess is that you can't use it in ways t…

The "ease" we used to refer to in Vitess primarily relates to its interaction with the application side, where it basically presents itself as "one big MySQL datastore". It uses a standard MySQL connector, and in general, once you have the infrastructure up and running with a compatible schema design, there's not too much to worry about from a coding standpoint. Sharding happens transparently to the application code, which generally translates to fewer code changes required.

Admittedly, that view left out the considerable challenge of actually deploying and running the infrastructure, designing and optimizing that schema, along with all the joys of managing large cluster environments.

That's what PlanetScale, the product, aims to solve. Dealing with clustering infrastructure IS a hurdle for most teams to overcome, and though Vitess' feature set and compatibility has expanded greatly to accommodate some of the most demanding use cases on the web today, a lot of its functionality can still be out of reach for a developer just trying to merge some code and a schema change.

Abstracting as much of that complexity away from the end user is the goal, as well as making their lives easier with a ton of the functionality we've always wanted to see built with Vitess. I can confirm that that is not an "easy" job on our end. :)

Re: PlanetScale – Database for Developers

#63
post #44

Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.

As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much be…

(crdb eng) I'm not sure what "document DB" means here, mind elaborating?

Re: PlanetScale – Database for Developers

#64
post #44

Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.

As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much be…

Something you really hit on here; there is no free lunch with clustered databases. You have to design your application to account for the sharding or you will run into data locality related performance issues.

Re: PlanetScale – Database for Developers

#65
The boulder-sized caveat for all this admittedly really neat stuff being:

> PlanetScale's Non-Blocking Schema Changes' workflow doesn't support FOREIGN KEYs in users' databases.

> PlanetScale determined that the production safety that Non-Blocking Schema Changes provide are worth this technical tradeoff. Learn more.

https://docs.planetscale.com/concepts/nonblocking-schema-cha...

Re: PlanetScale – Database for Developers

#66
post #44

Earlier quoted context omitted.

As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much be…

(crdb eng) I'm not sure what "document DB" means here, mind elaborating?

he is probably referring to the docdb document store in yugabyte: https://docs.yugabyte.com/latest/architecture/layered-archit...

Re: PlanetScale – Database for Developers

#67

Aren't schema changes are actually a big problem when there is a huge amount of data in the table. Adding a column / index to a table that has 500M rows in it is usually the pita. This is definitely something on the right direction, but would love to see this supporting this with the data itself.

That is exactly what we _do_ support. Our team includes the original developer of gh-ost for MySQL, which has been built to execute these kinds of massive changes at scale at GitHub. We've integrated it tightly with Vitess. Once your branch is ready to be merged with production, it executes that change in a completely non-blocking way.

Re: PlanetScale – Database for Developers

#68

Aren't schema changes are actually a big problem when there is a huge amount of data in the table. Adding a column / index to a table that has 500M rows in it is usually the pita. This is definitely something on the right direction, but would love to see this supporting this with the data itself.

That is exactly what we _do_ support. Our team includes the original developer of gh-ost for MySQL, which has been built to execute these kinds of massive changes at scale at GitHub. We've integrated it tightly with Vitess. Once your branch is ready to be merged with production, it executes that change in a completely non-blocking way.

Ok, but most of the time you would like to test that change on a copy or almost up to date replica, to see measure things like how long it does take. If you were copying the data, with things like PII filtering, to the branch development database and allow folks to test it there it would be even more amazing. Great start tho!

Re: PlanetScale – Database for Developers

#69

Earlier quoted context omitted.

(crdb eng) I'm not sure what "document DB" means here, mind elaborating?

he is probably referring to the docdb document store in yugabyte: https://docs.yugabyte.com/latest/architecture/layered-archit...

Indeed I was referring to yugabyte, apologies for the clumsy phrasing, I havent used crdb but I guess it is a postgres frontend layered on a KV store instead of a document store?

Re: PlanetScale – Database for Developers

#70
post #56

Really great finally seeing a “serverless” SQL database based on MySQL and Vitess that scales from 0 to n, even with a free tier! Where are you hosted? What latency should we expect from AWS, GCP, DigitalOcean and fly.io? And also what degree of compatibility should we expect with MySQL? The doc is quite sparse on this.

Vitess' compatibility with MySQL has made major leaps in the past couple of versions and the team has started focusing on locking in ongoing compatibility with various popular development frameworks. You can find those here, and more are getting added regularly: https://github.com/planetscale/vitess-framework-testing/

The basics of MySQL compatibility are described in here, though it's important to keep in mind that just because something "works" doesn't always mean it's the best way to do things in a sharded environment: https://vitess.io/docs/reference/compatibility/mysql-compati...

Post reply on HN