Reversible Migrations in Rails 3.1 • Hacker News Board

Re: Reversible Migrations in Rails 3.1

#21

Post by moe » Sat, May 07, 2011, 9:49 PM UTC

snprbob86 wrote: post #14

masklinn wrote: post #10

Earlier quoted context omitted.

> The database holds the authoritative schema. Why? > You shouldn't duplicate it in your code. If the database does not hold the authoritative schema, you're not duplicating anything. > What if you want to do something more complicated, like split the values in one column into two columns? You're going to need support for manual migrations anyway. Of course, but it's not like the generated migrations are special. The…

For one, the database is the only place where the entire schema is maintained. This includes indexes, constraints, relationships, etc. If I need to go add an index to a production system, that is not captured in the code. I should, however, dump the schema to a .sql file and check that in, but that's still separate from the model's .py file, so that file can't be considered authoritative. More importantly, the databa…

For one, the database is the only place where the entire schema is maintained.

That may be true for rails, but not for most other ORMs that don't follow the AR pattern. Said other ORMs take authority over the schema, which means by default they will sync the database to match your code - although the degree to which that is enforced is usually configurable.

If I need to go add an index to a production system

Then you will of course either test the change on a staging system first, or quickly backport the change after the hotfix. Whether the backport is done through a rails migration or a django model-change is not relevant to this discussion.

You might inherit a database/schema from someone else

Red herring. It's a rare case and people have managed this scenario with ORMs other than AR just fine since long before Rails was conceived.

Migration generation via diffing schemas? That's a much harder, in fact provably intractable problem!

A wide range of tools has a pretty good handle on this problem. (sqldelta.com, postfacto.org, xsql, red-gate.com, etc.)

Ambiguous changes are pretty rare during the evolution of your average webapp schema. Machines can very well solve 99% of cases on their own, and smartly interrogate the user for the rest.

Re: Reversible Migrations in Rails 3.1

#22

Post by moe » Sat, May 07, 2011, 9:53 PM UTC

snprbob86 wrote: post #19

moe wrote: post #18

Earlier quoted context omitted.

1) The database holds the authoritative schema. You shouldn't duplicate it in your code. That's nonsense, your application implies the authoritative schema as it relies on a given set of model fields to exist. More importantly any given version of your application may imply a different schema. Artificially detaching the schema from the code only causes problems and adds pointless complexity to the overwhelming majori…

When a migration runs, the schema.rb file is regenerated. This way, each checkin contains a complete snapshot of the schema that goes with that code. (side note: I think that schema.sql should be the default, not schema.rb)

Yes, that's what everybody refers to when they quickly need to look up a field.

However, having to look in two places to understand your model is a bit cumbersome, isn't it? Especially when complex associations come into play and you have to resolve the rails identifier magic in your head...

And then there's the question of why you should have to refer to a cached copy of your schema when you could just spell it all out in one place.

Re: Reversible Migrations in Rails 3.1

#23

Post by stretchwithme » Sat, May 07, 2011, 9:59 PM UTC

snprbob86 wrote: post #14

masklinn wrote: post #10

Earlier quoted context omitted.

> The database holds the authoritative schema. Why? > You shouldn't duplicate it in your code. If the database does not hold the authoritative schema, you're not duplicating anything. > What if you want to do something more complicated, like split the values in one column into two columns? You're going to need support for manual migrations anyway. Of course, but it's not like the generated migrations are special. The…

For one, the database is the only place where the entire schema is maintained. This includes indexes, constraints, relationships, etc. If I need to go add an index to a production system, that is not captured in the code. I should, however, dump the schema to a .sql file and check that in, but that's still separate from the model's .py file, so that file can't be considered authoritative. More importantly, the databa…

One nice thing about migrations describing your schema is that they can be used to migrate databases from different vendors. But if you rely upon scripts to create and migrate databases, you need to write a script for each one.

And there is no partial rollback. You get to write another script for that or do it manually.

In olden times, I used to have to write these for Oracle and SQL Server, and just keeping them in sync was a challenge. And you had to write a specific script or execute multiple smaller scripts to move from version 1.3 of the schema to version 3.9. The incremental, db agnostic, reversible approach is much better.

But you really need to make sure the end result is exactly what you expect before executing on production (and back it up first of course). The db-specific script seems much easier to trust, with no magic happening, especially when its atomic.

Re: Reversible Migrations in Rails 3.1

#24

Post by awj » Sat, May 07, 2011, 10:02 PM UTC

snprbob86 wrote: post #16

awj wrote: post #11

Earlier quoted context omitted.

Just to be contrary: > 1) The database holds the authoritative schema. You shouldn't duplicate it in your code. I care about the authoritative data model , which is slightly different from the authoritative schema . Beyond that, I don't edit the database schema. There are many cases where I'm happier to have a complete data model in one location versus (at least) two and needing to mentally parse how rails reacts to…

> I care about the authoritative data model Me too, but the truth about the world is that all inherit complexity is accompanied by incidental complexity :-/ You've got a database, you need to respect it's existence. > mentally parse how rails reacts to the database schema It's hilariously straightforward in 99% of cases: For each column, a getter and setter are created. The getter directly calls read_attribute and th…

FYI, you can re-declare as much of the model class as you need and simply call reset_column_info as a class method before using it. The models at work have some homemade serialized attributes (don't ask), and this makes it easier to deal with that situation.

Re: Reversible Migrations in Rails 3.1

#25

Post by prodigal_erik » Sat, May 07, 2011, 11:06 PM UTC

snprbob86 wrote: post #14

masklinn wrote: post #10

Earlier quoted context omitted.

> The database holds the authoritative schema. Why? > You shouldn't duplicate it in your code. If the database does not hold the authoritative schema, you're not duplicating anything. > What if you want to do something more complicated, like split the values in one column into two columns? You're going to need support for manual migrations anyway. Of course, but it's not like the generated migrations are special. The…

For one, the database is the only place where the entire schema is maintained. This includes indexes, constraints, relationships, etc. If I need to go add an index to a production system, that is not captured in the code. I should, however, dump the schema to a .sql file and check that in, but that's still separate from the model's .py file, so that file can't be considered authoritative. More importantly, the databa…

> More importantly, the database exists without the application.

This. I'm continually amazed how many people think they will only ever have one application talking to their database. Unless your project promptly crashes and burns, you're going to find yourself using tools in different languages, and it's going to be incredibly painful not to have invested in competent data modelling that avoids relying on some specific ORM.

Re: Reversible Migrations in Rails 3.1

#26

Post by jarin » Sat, May 07, 2011, 11:43 PM UTC

jarin wrote: post #6

snprbob86 wrote: post #5

Earlier quoted context omitted.

I too prefer Rail's approach. However, I disagree that Django's approach is equally valid :-) I've written about this before, but here's the short version: 1) The database holds the authoritative schema. You shouldn't duplicate it in your code. 2) What if you want to do something more complicated, like split the values in one column into two columns? You're going to need support for manual migrations anyway. 3) Inclu…

Well, yeah I totally agree. I didn't want to start a flame war :)

Mission failed.

Re: Reversible Migrations in Rails 3.1

#27

Post by foca » Sat, May 07, 2011, 11:46 PM UTC

jarin wrote: post #20

snprbob86 wrote: post #17

Earlier quoted context omitted.

He's making the same mistake I once made: Trying to use his models from the migrations. I'd really like to see Rails raise a deprecation warning if a model gets dynamically loaded by a migration....

Every Rails developer makes that mistake once (well, hopefully not more than once). What I'd really like to see next is a way to version seeds.rb.

git?

Re: Reversible Migrations in Rails 3.1

#28

Post by jarin » Sat, May 07, 2011, 11:52 PM UTC

foca wrote: post #27

jarin wrote: post #20

Earlier quoted context omitted.

Every Rails developer makes that mistake once (well, hopefully not more than once). What I'd really like to see next is a way to version seeds.rb.

git?

seeds.rb isn't intended to be run more than once on a particular database instance. Running it more than once results in duplicate entries, unless you're explicitly checking for dupes (not always possible).

It looks like seedbank is fairly close to what I want though, as it lets you maintain multiple sets of seeds: https://github.com/james2m/seedbank

Re: Reversible Migrations in Rails 3.1

#29

Post by fernandotakai » Sun, May 08, 2011, 12:05 AM UTC

snprbob86 wrote: post #15

fernandotakai wrote: post #8

Earlier quoted context omitted.

I have to say that i prefer the Django/sqlalchemy way. I know that i'm probably duplicating code, but with this way, i don't have to look at the database to know which properties a model has, i just have to go to the .py file. Also, i can use migrations as well to make sure the database is with the right version.

> i don't have to look at the database to know which properties a model has I just pop open db/schema.rb and then search for where the users table schema is described. This .rb file is automatically generated. In production systems, I actually configure it to use db/schema.sql which stores the SQL schema instead of the Ruby schema, since my production database has indexes that Rail's migrations can't describe directl…

I don't know, but i think that looking at models/yourmodel.py is easier. But well, this is me (and also, i came from a hibernate/spring background).

And for 'newbies', it's more intuitive to look for a class model than to look for sql schema.

Re: Reversible Migrations in Rails 3.1

#30

Post by moe » Sun, May 08, 2011, 12:16 AM UTC

prodigal_erik wrote: post #25

snprbob86 wrote: post #14

Earlier quoted context omitted.

For one, the database is the only place where the entire schema is maintained. This includes indexes, constraints, relationships, etc. If I need to go add an index to a production system, that is not captured in the code. I should, however, dump the schema to a .sql file and check that in, but that's still separate from the model's .py file, so that file can't be considered authoritative. More importantly, the databa…

> More importantly, the database exists without the application. This. I'm continually amazed how many people think they will only ever have one application talking to their database. Unless your project promptly crashes and burns, you're going to find yourself using tools in different languages, and it's going to be incredibly painful not to have invested in competent data modelling that avoids relying on some speci…

I repeat: Red Herring. Competent data modeling and AR migrations are in no way related.

Moreover most grown up applications consist of multiple database, since it would be unwise to mix e.g. reporting concerns into a webapp database schema.