Live data from Hacker News

Instant database clones with PostgreSQL 18

boringsql.com

131–140 of 168 posts

Re: Instant database clones with PostgreSQL 18

#131

Earlier quoted context omitted.

I was able to accomplish this by doing each test within its own transaction session that gets rolled-back after each test. This way I'm allowed to modify the database to suit my needs for each test, then it gets magically reset back to its known state for the next test. Transaction rollbacks are very quick.

Unfortunately a lot of our tests use transactions themselves because we lock the user row when we do anything to ensure consistency, and I'm pretty sure nested transactions are still not a thing.

You can emulate nested transactions using save points. A client uses that in production. And others in unit tests.

Re: Instant database clones with PostgreSQL 18

#132
post #130

PostgreSQL seems to have become the be-all, end-all SQL database that does everything and does it all well. And it's free! I'm wondering why anyone would want to use anything else at this point (for SQL).

PostgreSQL has no mature Vitess alternative. Hence, the largest oss OLTP database deployments tend to be MySQL. Like YouTube and Uber for example.

This is changing soon with Neki.

https://www.neki.dev

Re: Instant database clones with PostgreSQL 18

#133
post #130

Earlier quoted context omitted.

PostgreSQL has no mature Vitess alternative. Hence, the largest oss OLTP database deployments tend to be MySQL. Like YouTube and Uber for example.

This is changing soon with Neki. https://www.neki.dev

Nice. But it's not even available yet.

It will take years to call this mature. Certainly not "soon"

Re: Instant database clones with PostgreSQL 18

#135
post #130

Earlier quoted context omitted.

PostgreSQL has no mature Vitess alternative. Hence, the largest oss OLTP database deployments tend to be MySQL. Like YouTube and Uber for example.

This is changing soon with Neki. https://www.neki.dev

For me that page has basically 0 descriptive content.

Re: Instant database clones with PostgreSQL 18

#136
post #17

For those who can't wait for PG18 or need full instance isolation: I built Velo, which does instant branching using ZFS snapshots instead of reflinks. Works with any PG version today. Each branch is a fully isolated PostgreSQL container with its own port. ~2-5 seconds for a 100GB database. https://github.com/elitan/velo Main difference from PG18's approach: you get complete server isolation (useful for testing migrat…

You mean you told Claude a bunch of details and it built it for you? Mind you, I'm not saying it's bad per se. But shouldn't we be open and honest about this? I wonder if this is the new normal. Somebody says "I built Xyz" but then you realize it's vibe coded.

why does it matter?

Re: Instant database clones with PostgreSQL 18

#137
post #130

PostgreSQL seems to have become the be-all, end-all SQL database that does everything and does it all well. And it's free! I'm wondering why anyone would want to use anything else at this point (for SQL).

PostgreSQL has no mature Vitess alternative. Hence, the largest oss OLTP database deployments tend to be MySQL. Like YouTube and Uber for example.

YouTube has migrated mostly to Spanner from what I hear

Re: Instant database clones with PostgreSQL 18

#138

I set this up for my employer many years ago when they migrated to RDS. We kept bumping into issues on production migrations that would wreck things. I decided to do something about it. The steps were basically: 1. Clone the AWS RDS db - or spin up a new instance from a fresh backup. 2. Get the arn and from that the cname or public IP. 3. Plug that into the DB connection in your app 4. Run the migration on pseudo pro…

I love those "migration only fails in prod because of data quirks" bugs. They are the freaking worst. Have called off releases in the past because of it.

Re: Instant database clones with PostgreSQL 18

#139
post #17

For those who can't wait for PG18 or need full instance isolation: I built Velo, which does instant branching using ZFS snapshots instead of reflinks. Works with any PG version today. Each branch is a fully isolated PostgreSQL container with its own port. ~2-5 seconds for a 100GB database. https://github.com/elitan/velo Main difference from PG18's approach: you get complete server isolation (useful for testing migrat…

You, is an interesting word to use given that you plagiarized it.

Plagiarized from what? Happy to address if you can point to what you're referring to.

Re: Instant database clones with PostgreSQL 18

#140
post #17

For those who can't wait for PG18 or need full instance isolation: I built Velo, which does instant branching using ZFS snapshots instead of reflinks. Works with any PG version today. Each branch is a fully isolated PostgreSQL container with its own port. ~2-5 seconds for a 100GB database. https://github.com/elitan/velo Main difference from PG18's approach: you get complete server isolation (useful for testing migrat…

Hell yeah. I’ve been meaning to prototype this exact thing but with btrfs.

interesting, you have a link?
Post reply on HN