Live data from Hacker News

Migrating 1200 databases from MySQL to Postgres

psyenix.blogspot.com

41–50 of 85 posts

Re: Migrating 1200 databases from MySQL to Postgres

#41
If you are migrating to AWS RDS, I recommend AWS Data Migration service. I migrated my live database (~50GB) from Mysql to Postgres (both RDS) with zero downtime.

I used AWS Schema Conversion Tool for initial PG schema. I customized the generated schema for my specific needs.

Re: Migrating 1200 databases from MySQL to Postgres

#42
post #3

Would love to do this to the monstrosity I inherited in new job but it's a sispheyan task. One day.

You should see the one here. An old flat file database monstrosity from the 80s, imported into oracle, complete with all of the duplicated data. It's the most non relational, relational database ever.

I have 177 tables, 764 stored procedures and 44 joins.

Every single table has either a uniqid() or a UUID() which isn't the primary key (they all have incrementing integer keys) but is 'joined' on in code.

When he needs to 'join' he used coalesce() on selects.

Referential integrity doesn't exist.

Oh and the cherry on the shit sandwhich, we have four seperate systems that talk to this database written in 3 different languages (Java (Android scanners), C# (Factory scanners) and PHP (the main system).

So there is no way to accurately know which SP's are used by what without grepping the entire codebase looking for call .

The PHP is written badly in the old PHP pure procedural style running on an outdated version of PHP/Debian/MySQL.

Basically if you took the absolute worst approach to everything this would be the end result.

It's good fun!.

Re: Migrating 1200 databases from MySQL to Postgres

#43
post #28

Is it an universally accepted truth now that it's generally better to use Postgres then to use MySql? Would anybody here use MySql for a new project? If so, why?

Nope, because using MySQL instead of Postgres essentially cripples your dev tools once you know about everything PG opens up for you.

http://www.brightball.com/articles/why-should-you-learn-post...

Re: Migrating 1200 databases from MySQL to Postgres

#44

Earlier quoted context omitted.

What storage engines and options do you use?

XtraDB if you can, InnoDB if you can't.

FWIW, MariaDB 10.2 no longer defaults to XtraDB, now considering InnoDB mostly on-par. [0]

[0] https://mariadb.com/kb/en/mariadb/why-does-mariadb-102-use-i...

Re: Migrating 1200 databases from MySQL to Postgres

#45
post #31

Earlier quoted context omitted.

Ditto. I still choose it for small to medium sized webapps because - let's face it - the data storage is not their pain point. MySQL is fast enough, usually has enough features, I know how to tune it and queries, and I normally only bump into 2-3 things in a project which would've been easier with PostgreSQL. And can work around them. The backwards-incompatible SQL_MODE changes between MySQL 5.6 and 5.7 made me recon…

> The backwards-incompatible SQL_MODE changes between MySQL 5.6 and 5.7 made me reconsider, as legacy apps needed updating. Uhm, err, that change is to stop the database from silently trowing away your data . It's not about "fast enough", or "easy". It's about your database actually saving your data . If you're happy for your "database" to keep quietly throwing away data in order to keep legacy apps "working", you ca…

It's not just changes to stop data being thrown away. This issue has been painful: https://github.com/doctrine/doctrine2/pull/6143

Across multiple businesses, they've upgraded servers and got MySQL 5.7 installed, only to wonder why the app has stopped working. People didn't expect that from a point release/the default new operating system package. We could argue the point, but it's the expectation of businesses vs software engineers.

Re: Migrating 1200 databases from MySQL to Postgres

#46
post #25
post #23

Earlier quoted context omitted.

It appears work is active (May 2017) in this area, but accent-insensitive collations are still a ways off: https://blog.2ndquadrant.com/icu-support-postgresql-10/ edit: fulafel corrects me below that you can get accent-insensitive collations working using PostgreSQL 10's new ICU collation support. Note that PostgreSQL 10 is currently pre-release.

That post says that they currently work, but there is a new os-independent collation method coming. Edit re your edit: the already working collations are in PG 9.x - the OS independent collations are coming in 10.x. (Hence the post title, "More robust collations with ICU support in PostgreSQL 10")

But do these "already working" collations include accent-insensitive variants? I didn't think so, and I don't think they are coming in PostgreSQL 10 either. Do you have other information on this?

Re: Migrating 1200 databases from MySQL to Postgres

#47
post #39
post #28

Is it an universally accepted truth now that it's generally better to use Postgres then to use MySql? Would anybody here use MySql for a new project? If so, why?

Like other comments here state, in general, use what you know. That said, postgresql has ~always been the better rdbms . Some would argue mysql at times have been a better ~key-value store with ~sql interface. Apparently postgresql is difficult to run in a properly restricted bsd jail. Other than that, I'd be hard pressed to come up with reasons to prefer mysql on technical merit over either postgresql, or, say redis…

I'm running postgresql in a FreeBSD vnet jail without issue. The network stack is not shared as it's a vnet jail. I have it set for per jail sysvipc, new in FreeBSD 11, which was the last thing I needed to fully isolate it from other jails.

Is there anything else that's needed to properly restrict the jail?

Re: Migrating 1200 databases from MySQL to Postgres

#49
I'm not against the idea of using postgres, but I have a sneaky suspicion, that you could have gotten that 30% speed benefit from MySQL, if you did some performance work on it. Or in other words, I don't feel that migrating from one relational database to another similar relational database was worth that much trouble.
Post reply on HN