Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using the expand and contract pattern for schema changes

prisma.io

Post reply on HN
Print view View original

21–30 of 45 posts

Re: Using the expand and contract pattern for schema changes

#21

Post by tczMUFlmoNk »

leetrout wrote: post #2

I use this example when I speak about and teach devops trainings. I call it the migration sandwich. (Nothing to do with the cube rule). A piece of bread isn't a sandwich and a single migration in a tool like alembic isn't a "sandwich" either. You have a couple layers of bread with one or several layers of toppings and it's not a sandwich until it's all done. People get a laugh out of the "idiot sandwich meme" and we…

I have usually heard it called "A–AB–B migrations". As in, you support version A, then you support both version A and version B, then you support just version B.

The rest of the sequencing details follow from this idea.

Re: Using the expand and contract pattern for schema changes

#23

Post by marcosdumay »

mcdonje wrote: post #14
leetrout wrote: post #2

I use this example when I speak about and teach devops trainings. I call it the migration sandwich. (Nothing to do with the cube rule). A piece of bread isn't a sandwich and a single migration in a tool like alembic isn't a "sandwich" either. You have a couple layers of bread with one or several layers of toppings and it's not a sandwich until it's all done. People get a laugh out of the "idiot sandwich meme" and we…

This is the same pattern as versioning, but with an extremely short sunset for the old version.

It's actually not.

Versioning is a concept where each version lives in non-intersecting time intervals.

This concept is completely focusing on the fact that your structures lifetimes must absolutely have non-empty intersections. It's close to the opposite.

Re: Using the expand and contract pattern for schema changes

#24

Post by chrisweekly »

davedx wrote: post #17

I use Prisma on almost all my node.js projects these days, and I wish that part of schema migrations was also automated by Prisma. But last I checked, it doesn't even rename columns properly. I feel like maybe they should invest more R&D in their migrations technology? The ORM is pretty great.

Lack of reasonable support for migrations turned me off to Prisma when I first encountered it (in a KCDodds Remix app circa 2021-ish). I'm surprised that's still unaddressed.

Re: Using the expand and contract pattern for schema changes

#25

Post by marcosdumay »

isuckatcoding wrote: post #16

Ok hear me out. What if this whole process was statefully managed for you as an add on to your database? Like you essentially defined the steps in a temporal like workflow and then it does all the work of expanding, verifying and contracting.

If you solve that "verifying" step, you will already revolutionize software development.

Re: Using the expand and contract pattern for schema changes

#26

Post by vvern »

isuckatcoding wrote: post #16

Ok hear me out. What if this whole process was statefully managed for you as an add on to your database? Like you essentially defined the steps in a temporal like workflow and then it does all the work of expanding, verifying and contracting.

It should be this way. Clients should have some protocol to communicate the schema they expect to the database probably with some versioning scheme. The database should be able to serve multiple mutually compatible views over the schema (stay robust to column renames for example). The database should manage and prevent the destruction of in use views of that schema. After an old view has been made incompatible, old clients needing that view should be locked out.

Re: Using the expand and contract pattern for schema changes

#27

Post by manmal »

sarchertech wrote: post #6

This is just the natural solution that falls out if you want to change a schema with no downtime. I always just called it “dual writing”.

In the step where writes are added, from the client to the new schema instance, how does this work when writes depend on existing data, when the existing data hasn’t even been copied? Constrains like foreign keys will prevent this from working, no?

Re: Using the expand and contract pattern for schema changes

#28

Post by chrisweekly »

marcosdumay wrote: post #23
mcdonje wrote: post #14

Earlier quoted context omitted.

This is the same pattern as versioning, but with an extremely short sunset for the old version.

It's actually not. Versioning is a concept where each version lives in non-intersecting time intervals. This concept is completely focusing on the fact that your structures lifetimes must absolutely have non-empty intersections. It's close to the opposite.

> "Versioning is a concept where each version lives in non-intersecting time intervals."

Is it? Node.js publishes "Current", "LTS" and "Maintenance" versions, and there's always a reasonable time interval during which consumers typically upgrade from eg Maintenance to newer LTS or even Current. From the publishing side, that's very similar to "expand and contract", in temporarily expanding what's supported to include Current, and dropping support for oldest versions leaving Maintenance. It's continuous instead of ad hoc, and there are more than 2 versions involved, but the principle is basically the same (at least if you squint).

Though I guess if you're talking strictly about schema management strategies, then yeah, "versioning" might be very different from "expand contract", as you noted.

Re: Using the expand and contract pattern for schema changes

#29

Post by sarchertech »

manmal wrote: post #27
sarchertech wrote: post #6

This is just the natural solution that falls out if you want to change a schema with no downtime. I always just called it “dual writing”.

In the step where writes are added, from the client to the new schema instance, how does this work when writes depend on existing data, when the existing data hasn’t even been copied? Constrains like foreign keys will prevent this from working, no?

You wouldn’t enable foreign key constraints until you finish backfilling the old data in that case. Or you do it in phases where you do all of these steps and migrate the dependency first.

Re: Using the expand and contract pattern for schema changes

#30

Post by sarchertech »

evancordell wrote: post #15
sarchertech wrote: post #6

This is just the natural solution that falls out if you want to change a schema with no downtime. I always just called it “dual writing”.

We always called these "four-phase migrations". An old Stripe article used similar naming[0]. [0]: https://stripe.com/blog/online-migrations

I’ve heard that one too. I think the key insight is that you need to “stop the bleeding” (stop creating more old data that needs to be migrated) before you do any backfilling. That’s why I always called it dual writing because that’s the stop the bleeding step.
Post reply on HN