Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PlanetScale for Postgres is now GA

planetscale.com

Post reply on HN
Print view View original

151–160 of 196 posts

Re: PlanetScale for Postgres is now GA

#151

Post by sgarland »

whizzter wrote: post #105
cyberpunk wrote: post #88

Earlier quoted context omitted.

This isn’t any kind of answer it’s a bunch of non-statements.. How is this any different that rds on nvme disks? With a name like planet scale i assumed it would be some multi-master setup?

sharded setup with a bit fast and loose foreign key management, so very good for performance but not a drop-in replacement if you rely on your foreign keys to be constrained/checked by the database.

So perfect for most web dev companies, then.

“We handle FKs in the app for flexibility.”

“And how many orphaned rows do you have?”

“…”

Re: PlanetScale for Postgres is now GA

#152

Post by sgarland »

tracker1 wrote: post #126
n_u wrote: post #65

Earlier quoted context omitted.

From my position MySQL pros: The MySQL docs on how the default storage engine InnoDB locks rows to support transaction isolation levels is fantastic. [1] This can help you better architect your system to avoid lock contention or understand why existing queries may be contending for locks. As far as I know Postgres does not have docs like that. MySQL uses direct I/O so it disables the OS page cache and uses its own bu…

To me, every time I've touched MySQL, I've found an issue that just rubbed me the wrong way... starting with the fact that UTF8 isn't, and even across new major versions hasn't changed to an alias for the real UTF8. (VAR)BINARY does case sensitive collation based on db default, it shouldn't do any type of textual comparison even if the data is "text". You can't do foreign keys with ANSI quotes for the table/field nam…

> (VAR)BINARY

Close [0]: (VAR)CHAR BINARY, which is its own type, uses the `_bin` collation for the column or table character set. (VAR)BINARY stores binary strings, and uses the `binary` character set and collation.

In fairness, the amount of options, gotchas, and WTF involved with collations in any DB is mind-boggling.

[0]: https://dev.mysql.com/doc/refman/8.4/en/binary-varbinary.htm...

Re: PlanetScale for Postgres is now GA

#153

Post by ndriscoll »

parthdesai wrote: post #147
ndriscoll wrote: post #123

Earlier quoted context omitted.

> Businesses that need to haul ass > Benchmarks are done on a dual-core VM with "unlimited" IOPS I'd be interested in a comparison with a pair of Beelink SER5 Pros ($300 each) in master-slave config.

> > Benchmarks are done on a dual-core VM with "unlimited" IOPS Unlimited is a feature here, no need to be snarky. They famously went against the accepted practice of separating storage from compute, and as a result, you reduce latency by an order of magnitude and get unlimited IOPS.

You do not get unlimited IOPS with any technology, but you especially do not get it in AWS, where the machines seem to be? Writing "unlimited" is completely unserious. If it's 67k read/33k write at 4k qd32 or something just say so. Or if you're actually getting full bandwidth to a disk with a 2 core VM (doubt), say 1.5M or whatever.

Re: PlanetScale for Postgres is now GA

#154

Post by mattrobenolt »

ndriscoll wrote: post #153
parthdesai wrote: post #147

Earlier quoted context omitted.

> > Benchmarks are done on a dual-core VM with "unlimited" IOPS Unlimited is a feature here, no need to be snarky. They famously went against the accepted practice of separating storage from compute, and as a result, you reduce latency by an order of magnitude and get unlimited IOPS.

You do not get unlimited IOPS with any technology, but you especially do not get it in AWS, where the machines seem to be? Writing "unlimited" is completely unserious. If it's 67k read/33k write at 4k qd32 or something just say so. Or if you're actually getting full bandwidth to a disk with a 2 core VM (doubt), say 1.5M or whatever.

Unlimited in this context just means you're going to be CPU limited before you hit limits on IOPS. It'd be technically not possible to be bottlenecked on IOPS.

That might not be 100% true, but I've never seen a RDBMS be able to saturate IOPS on a local NVMe. It's some quite specialized software to leverage every ounce of IOPS without being CPU bottlenecked first. Postgres and MySQL are not it.

Re: PlanetScale for Postgres is now GA

#155

Post by ndriscoll »

mattrobenolt wrote: post #154
ndriscoll wrote: post #153

Earlier quoted context omitted.

You do not get unlimited IOPS with any technology, but you especially do not get it in AWS, where the machines seem to be? Writing "unlimited" is completely unserious. If it's 67k read/33k write at 4k qd32 or something just say so. Or if you're actually getting full bandwidth to a disk with a 2 core VM (doubt), say 1.5M or whatever.

Unlimited in this context just means you're going to be CPU limited before you hit limits on IOPS. It'd be technically not possible to be bottlenecked on IOPS. That might not be 100% true, but I've never seen a RDBMS be able to saturate IOPS on a local NVMe. It's some quite specialized software to leverage every ounce of IOPS without being CPU bottlenecked first. Postgres and MySQL are not it.

What does "local NVMe" mean for you? AFAIK in AWS if you have a 2 core VM you're getting ~3% of a single disk worth of IOPS for their attached storage. Technically NVMe. Not generally what people think when a laptop can do 50x more IO. The minipc I mentioned has 4x the core count and... well who knows how much more IO capacity, but it seems like it should be able to trounce both. Obviously an even more interesting comparison would be... a real server. Why is a database company running benchmarks on something comparable to my low-end phone?

Anyway, saying unlimited is absurd. If you think it's more than you need, say how much it is and say that's more than you need. If you have infinite IOPS why not do the benchmark on a dataset that fits in CPU cache?

Re: PlanetScale for Postgres is now GA

#156

Post by rcrowley »

cyberpunk wrote: post #88
brettgriffin wrote: post #85

Earlier quoted context omitted.

Literally the first line of every line on the site: > PlanetScale is the world’s fastest relational database platform. We offer PostgreSQL and Vitess databases that run on NVMe-backed nodes to bring you scale, performance, reliability, and cost-efficiencies — without sacrificing developer experience. > PlanetScale is a relational database platform that brings you scale, performance, and reliability — without sacrific…

This isn’t any kind of answer it’s a bunch of non-statements.. How is this any different that rds on nvme disks? With a name like planet scale i assumed it would be some multi-master setup?

The short answer is that RDS doesn't run on local NVMe disks, it runs on EBS.

Re: PlanetScale for Postgres is now GA

#157

Post by mattrobenolt »

bourbonproof wrote: post #146
mattrobenolt wrote: post #50

Earlier quoted context omitted.

We deal with this by always running 3 nodes in a cluster, one per AZ, and strong backup/restore processes. So yes, the data per-node is ephemeral, but it is redundant and durable for the whole cluster.

Do I understand this right: if these 3 nodes shutdown for some reason, all data is lost and you have to actually restore from backup instead of just starting the machine again. And even if you have to restart one node (due to updates, or crashes) you also have to restore from backup? If so, why not pick a hosting provider that doesn't wipe the disk when machine shuts down?

It's more than just shutting down. You'd have to have an actual failure. Data isn't lost on a simple restart. It'd require 3 nodes to die in 3 different AZs.

While that's not impossible, the reality is that's very low.

So simply restarting nodes wouldn't trigger restoring from backup, but yes, in our case, replacing nodes entirely does require that node to restore from a backup/WALs and catch back up in replication.

EBS doesn't entirely just solve this, you still have failures and still need/want to restore from backups. This is built into our product as a fundamental feature. It's transparent to users, but the upside is that restoring from backups and creating backups is tested every day multiple times per day for a database. We aren't afraid of restoring from backups and replacing nodes by choice or by failure. It's the same to us.

We do all of the same operations already on EBS. This magic is what enables us to be able to use NVMe's since we treat EBS as ephemeral already.

Re: PlanetScale for Postgres is now GA

#158

Post by rcrowley »

n_u wrote: post #131
rcrowley wrote: post #96

Earlier quoted context omitted.

1. I don't know if there's a single name for this. I will point out that AWS EBS and Google Persistent Disk as industrial examples of distributed, replicated block devices are also providing durability via replication. They're just providing it at a lower level that ends up sacrificing performance. I'm struggling to come up with a citation but I think it's either Liskov or Lynch that offered a proof to the effect of…

P(two failures within MTTR for first node) = P(one failure)P(second failure within MTTR of first node|one failure) independence simplifies things = P(one failure)P(second failure within MTTR of first node) = P(one failure) * (1 - e^-λx) where x = MTTR for first node λ = 1/MTBF plugging in the numbers from your blog post P(one failure within 30 days) = 0.01 not sure if this part is correct. MTTR = 5 minutes + 5 hours…

I computed P(node failure within MTTR) = 0.00007 same as you. I extrapolated this to the outage scenario P(at least two node failures within MTTR) = P(node failure within MTTR)^2 * (1-P(node failure within MTTR)) + P(node failure within MTTR)^3 = 5.09 * 10^-9 which rounds to 0.0000001%.

Re: PlanetScale for Postgres is now GA

#159

Post by parthdesai »

ndriscoll wrote: post #155
mattrobenolt wrote: post #154

Earlier quoted context omitted.

Unlimited in this context just means you're going to be CPU limited before you hit limits on IOPS. It'd be technically not possible to be bottlenecked on IOPS. That might not be 100% true, but I've never seen a RDBMS be able to saturate IOPS on a local NVMe. It's some quite specialized software to leverage every ounce of IOPS without being CPU bottlenecked first. Postgres and MySQL are not it.

What does "local NVMe" mean for you? AFAIK in AWS if you have a 2 core VM you're getting ~3% of a single disk worth of IOPS for their attached storage. Technically NVMe. Not generally what people think when a laptop can do 50x more IO. The minipc I mentioned has 4x the core count and... well who knows how much more IO capacity, but it seems like it should be able to trounce both. Obviously an even more interesting co…

They literally state this on their metal offering

> Unlimited I/O — Metal's local NVMe drives offer higher I/O bandwidth than network-attached storage. You will run out of CPU long before you use all your I/O bandwidth.

https://planetscale.com/metal#benefits-of-metal

Re: PlanetScale for Postgres is now GA

#160

Post by mattrobenolt »

ndriscoll wrote: post #155
mattrobenolt wrote: post #154

Earlier quoted context omitted.

Unlimited in this context just means you're going to be CPU limited before you hit limits on IOPS. It'd be technically not possible to be bottlenecked on IOPS. That might not be 100% true, but I've never seen a RDBMS be able to saturate IOPS on a local NVMe. It's some quite specialized software to leverage every ounce of IOPS without being CPU bottlenecked first. Postgres and MySQL are not it.

What does "local NVMe" mean for you? AFAIK in AWS if you have a 2 core VM you're getting ~3% of a single disk worth of IOPS for their attached storage. Technically NVMe. Not generally what people think when a laptop can do 50x more IO. The minipc I mentioned has 4x the core count and... well who knows how much more IO capacity, but it seems like it should be able to trounce both. Obviously an even more interesting co…

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-inst...

Not all AWS instance types support NVMe drives. It's not the same as normal attached storage.

I'm not really sure your arguments are in good faith here tho.

This is just not a configuration you can trivially do while maintaining durability and HA.

There's a lot of hype going the exact opposite direction and more separation of storage and compute. This is our counter to that. We think even EBS is bad.

This isn't a setup that is naturally just going to beat a "real server" that also has local NVMes or whatever you'd do yourself. This is just not what things like RDS or Aurora do, etc. Most things rely on EBS which is significantly worse than local storages. We aren't really claiming we've invented something new here. It's just unique in the managed database space.

Post reply on HN