Migrations and Future Proofing • Hacker News Board

Re: Migrations and Future Proofing

#11

Post by shykes » Thu, Sep 25, 2014, 12:41 AM UTC

When we introduced pluggable storage drivers in Docker 0.7, we wanted all existing data to work as usual (full compatibility of data at rest), but we also wanted to migrate the layout of the legacy storage system (based on AUFS) so that it would be "just another driver" instead of a perpetual special case. At the same time, we didn't have the luxury of a full-stop mandatory migration, because if anything went wrong, the upgrade would fail and the user would be stuck in a hairy half-migrated situation. Keep in mind we are not talking about a relational database, but directories used to mount the root filesystems of live containers. That means that some of those directories may be mounted and therefore unmovable. So we had to accomodate partial migration failure, and the possibility of a partially migrated install.

So we shipped a migration routine which ran at startup every time and gave up (gracefully and atomically) at the slightest sign of trouble. Over time, we reasoned, each install would converge towards full migration, and the huge majority of containers would be migrated within seconds of the upgrade. The rest would be much easier to deal with if anybody had any trouble.

Of course we had the luxury of a data structure which allowed this.

Re: Migrations and Future Proofing

#12

Post by jamii » Thu, Sep 25, 2014, 1:02 AM UTC

tieTYT wrote: post #5

How does Erlang deal with these problems? It often touts minimal downtime and the ability to run updates to your code while it's running. I think that means you can have Process V1 and Process V2 running on the same server simultaneously. If they read from the same database, won't you run into issues?

When you update an erlang module, the vm stores the old and new versions of the code. Calls like foo() call the current running version. Calls like module:foo() call the latest version of that module. So typically you would have all the control flow for a given process in one module and it would use the module call to control when the upgrade happens. At that point it gets to pause and migrate it's data.

The process isolation and code swap mechanisms do give you some help, but when it comes to messages sent between processes you are back in the same boat with API versioning or carefully ordered upgrades.

There is no real magic involved. It takes careful thought and tons of testing to make live upgrades work. Most of the projects I worked on preferred to just eat the few seconds downtime instead rather than risk getting the server into some weird in-between state.

Re: Migrations and Future Proofing

#13

Post by ivanceras » Wed, Oct 01, 2014, 5:37 PM UTC

w01fe wrote: post #8

Author here, would love feedback on this, and also happy to answer any questions.

I saved the document for later use, but it got me thinking how ORMs like hibernate is dealing with the migration and future proofing issues. I wrote code libraries for a small company, in which part of the libraries design is to allow data transformation/representation of Data Models to work on the new database schema/changes while representing the data structure as if it were from previous versions. So I ended up writing generators to create a mapping of the DAO and the Models exposed to the API's in such a way that when the database changes while you still have to support the previous version of the API, I will only have to edit/tweak the mappers of the previous version of the API.

The library has been opensourced here in this link https://github.com/ivanceras/orm