Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PlanetScale for Postgres is now GA

planetscale.com

Post reply on HN
Print view View original

161–170 of 196 posts

Re: PlanetScale for Postgres is now GA

#161

Post by rcrowley »

sgarland wrote: post #151
whizzter wrote: post #105

Earlier quoted context omitted.

sharded setup with a bit fast and loose foreign key management, so very good for performance but not a drop-in replacement if you rely on your foreign keys to be constrained/checked by the database.

So perfect for most web dev companies, then. “We handle FKs in the app for flexibility.” “And how many orphaned rows do you have?” “…”

The question isn't how many orphaned rows do you have, it's whether it matters. Databases are wonderful but they cannot maintain every invariant and they cannot express a whole application. They're one tool in the belt.

Re: PlanetScale for Postgres is now GA

#162

Post by ndriscoll »

mattrobenolt wrote: post #160
ndriscoll wrote: post #155

Earlier quoted context omitted.

What does "local NVMe" mean for you? AFAIK in AWS if you have a 2 core VM you're getting ~3% of a single disk worth of IOPS for their attached storage. Technically NVMe. Not generally what people think when a laptop can do 50x more IO. The minipc I mentioned has 4x the core count and... well who knows how much more IO capacity, but it seems like it should be able to trounce both. Obviously an even more interesting co…

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-inst... Not all AWS instance types support NVMe drives. It's not the same as normal attached storage. I'm not really sure your arguments are in good faith here tho. This is just not a configuration you can trivially do while maintaining durability and HA. There's a lot of hype going the exact opposite direction and more separation of storage and compute. This is…

Right, but as far as I know, the only instances that give you full bandwidth with NVMe drives are metal. Everything else gives you a proportional fraction based on VM size. So for most developers, yes it is hard to saturate an NVMe drive with e.g. 8-16 cores. Now how about the 100+ cores you actually need to rent to get that full bandwidth?

I agree that EBS and the defaults RDS tries to push you into are awful for a database in any case. 3k IOPS or something absurd like that. But that's kind of the point: AWS sells that as "SSD" storage. Sure it's SSD, but it's also 100-1000x slower than the SSDs most devs would think of. Their local "NVMe" is AFAIK also way slower than what it's meant to evoke in your mind unless you're getting the largest instances.

Actually, showing scaling behavior with large instances might make Planetscale look even better than competitors in AWS if you can scale further vertically before needing to go horizontal.

Re: PlanetScale for Postgres is now GA

#163

Post by sgarland »

rcrowley wrote: post #161
sgarland wrote: post #151

Earlier quoted context omitted.

So perfect for most web dev companies, then. “We handle FKs in the app for flexibility.” “And how many orphaned rows do you have?” “…”

The question isn't how many orphaned rows do you have, it's whether it matters. Databases are wonderful but they cannot maintain every invariant and they cannot express a whole application. They're one tool in the belt.

> cannot express a whole application

Not with that attitude: https://docs.postgrest.org/en/v13/index.html

Orphaned rows can very much matter for data privacy concerns, which is also where I most frequently see this approach failing.

Re: PlanetScale for Postgres is now GA

#165

Post by dangoodmanUT »

petergeoghegan wrote: post #36
dangoodmanUT wrote: post #6

Earlier quoted context omitted.

Postgres (esoterically?) has some issues with index bloat on high-insert workloads, does PlanetScale do anything special to tune for this by default, since it caters to higher-perf workloads (over something like supabase)?

Can you provide more detail/a reference? I've done extensive work on improving the Postgres B-Tree code, over quite a number of releases. I'm not aware of any problems with high-insert workloads in particular. I have personally fixed a number of subtle issues that could lead to lower space utilization with such workloads [1][2] in the past, though. if there's a remaining problem in this area, then I'd very much like…

In a previous use case, when using postgres as a WAL-like append only store, I noticed that indexes would get massive. Then, after a while, they'd magically shrink. I had eventually switched to an API on top of Badger (golang KV), which afforded me an order of magnitude lower latency at ~30% of the resources IIRC. I'm sure there might have been some tuning I could have done to improve it.

I've also heard similar behaviors exhibited from other folks who had similar high-write workloads on postgres.

Sorry, I don't have anything super tangible to provide off the top of my head, or metrics/code I can share to recreate! It was also a project that required a lot of data to recreate the setup for.

Re: PlanetScale for Postgres is now GA

#166

Post by 0x6c6f6c »

candiddevmike wrote: post #61
dfee wrote: post #29

Earlier quoted context omitted.

i'll take the opposite side. i was very impressed with their website. the very first line: > The world’s fastest and most scalable cloud databases the second line: > PlanetScale brings you the fastest databases available in the cloud. Both our Postgres and Vitess databases deliver exceptional speed and reliability, with Vitess adding ultra scalability through horizontal sharding. i know exactly what they do. zero flu…

Baseless marketing claims aren't considered fluff?

It's fluff to give the elevator pitch now guys, be warned

Re: PlanetScale for Postgres is now GA

#167

Post by sreekanth850 »

theanirudh wrote: post #49
benterix wrote: post #40

Earlier quoted context omitted.

Out of curiosity: how do you connect your databases to external services that are consuming these data? In places I do similar work, databases are usually in the same private network as the instances which are reading and writing data to them. If you put them somewhere on the internet, apart from security, doesn't it affect latency?

Their databases are hosted on AWS and GCP so latency isn't much of an issue. They also have AWS Private Link and if configured it won't go over the internet.

No matter if its hosted on Azure GCP or AWS, latency is real. Cloud providers doesn't magically eliminates the Geography and phhysics. Private network don't eliminates latency magically. In general, Any small latency hike can potentially create performance bottlenecks for write operations in strong consistency DB like postgres or MySQL because each write operation go through a round trip from your server to remote planetscale server that create transaction overhead. Complex transactions with multiple statements can amplify this latency due to this round trip. But you could potentially reduce this latency by hosting your app near to where planet scale host their DB cluster though. But that is a dependency or compromise. Edit: A few writes per second? Probably fine. Hundreds of writes per second? Those extra milliseconds become a real bottleneck.

Re: PlanetScale for Postgres is now GA

#168

Post by mattrobenolt »

sreekanth850 wrote: post #167
theanirudh wrote: post #49

Earlier quoted context omitted.

Their databases are hosted on AWS and GCP so latency isn't much of an issue. They also have AWS Private Link and if configured it won't go over the internet.

No matter if its hosted on Azure GCP or AWS, latency is real. Cloud providers doesn't magically eliminates the Geography and phhysics. Private network don't eliminates latency magically. In general, Any small latency hike can potentially create performance bottlenecks for write operations in strong consistency DB like postgres or MySQL because each write operation go through a round trip from your server to remote pl…

You can simply place your database in the same AWS or GCP region and the same AZs.

Re: PlanetScale for Postgres is now GA

#169

Post by mattrobenolt »

ndriscoll wrote: post #162
mattrobenolt wrote: post #160

Earlier quoted context omitted.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-inst... Not all AWS instance types support NVMe drives. It's not the same as normal attached storage. I'm not really sure your arguments are in good faith here tho. This is just not a configuration you can trivially do while maintaining durability and HA. There's a lot of hype going the exact opposite direction and more separation of storage and compute. This is…

Right, but as far as I know, the only instances that give you full bandwidth with NVMe drives are metal. Everything else gives you a proportional fraction based on VM size. So for most developers, yes it is hard to saturate an NVMe drive with e.g. 8-16 cores. Now how about the 100+ cores you actually need to rent to get that full bandwidth? I agree that EBS and the defaults RDS tries to push you into are awful for a…

Right, but I think you're kinda missing a lot of the tangible benefits here. This IMO is just reinforcing the idea of "unlimited" IOPS. You can't physically use the totality of IOPS available on the drives.

Even if you can't saturate them, even with low CPU cores, latency is drastically better which is highly important for database performance.

Having low latency is tangibly more important than throughput or number of IOPS once your dataset is larger than RAM no matter how many CPU cores you have.

Chasing down p95s and above really shine with NVMes purely from having whatever order of magnitude less latency.

Less latency also equates to less iowait time. All of this just leads to better CPU time utilization on your database.

Re: PlanetScale for Postgres is now GA

#170

Post by petergeoghegan »

dangoodmanUT wrote: post #165
petergeoghegan wrote: post #36

Earlier quoted context omitted.

Can you provide more detail/a reference? I've done extensive work on improving the Postgres B-Tree code, over quite a number of releases. I'm not aware of any problems with high-insert workloads in particular. I have personally fixed a number of subtle issues that could lead to lower space utilization with such workloads [1][2] in the past, though. if there's a remaining problem in this area, then I'd very much like…

In a previous use case, when using postgres as a WAL-like append only store, I noticed that indexes would get massive. Then, after a while, they'd magically shrink. I had eventually switched to an API on top of Badger (golang KV), which afforded me an order of magnitude lower latency at ~30% of the resources IIRC. I'm sure there might have been some tuning I could have done to improve it. I've also heard similar beha…

> In a previous use case, when using postgres as a WAL-like append only store, I noticed that indexes would get massive. Then, after a while, they'd magically shrink.

It's possible to recycle pages within indexes that have some churn (e.g., with workloads that use bulk range deletions). But it's not possible for indexes to shrink on their own, in a way that can be observed by monitoring the output of psql's "\di+" command. For that you'd need to REINDEX or run VACUUM FULL.

Post reply on HN