Live data from Hacker News

AWS Database Migration Service

aws.amazon.com

51–60 of 67 posts

Re: AWS Database Migration Service

#51
post #42

Earlier quoted context omitted.

I wonder if you could realistically do a two stage migration, X -> Amazon -> Y

Amazon probably wouldn't care if you did. They still get paid to shuffle the bits.

What they really want is keep you on RDS, not just get paid for the migration bandwidth.

Re: AWS Database Migration Service

#52
post #39

Earlier quoted context omitted.

You can snapshot your RDS database and restore it as an Aurora DB. Only caveat is it has to be MySQL 5.6. http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora...

Not quite the same, unless I'm misunderstanding. I'm talking about an online migration on a single RDS instance with an engine switch handled fully behind the scenes. The link provided doesn't seem to handle data changes since the snapshot, and also requires a switch in the application to point to the new instance which has been restored via snapshot. DMS gets closer than the above link, I think, since it will handle…

The on disk format for Aurua is likely different.

Re: AWS Database Migration Service

#53

One of the biggest problems I've had with RDS (postgres specifically) is not being able to do incremental offsite backups. I wonder if the database migration service can help with that, as they say the process is "reliable" and "automatically restarts the process and continues the migration from where it was halted"

[deleted]

Re: AWS Database Migration Service

#54
post #43

Does anyone know how the MySQL migration & replication works under the hood? How is Amazon doing all this remotely via just a DB connector?

Typical MySQL replication works when slave is listening to changes on master's binlog. Which, essentially, is a log of all operations performed on the dataset. I did some quick googling and apparently it can be accessed quite easily: http://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html so I'd assume that this is how they are doing the replication, or something very similar.

As for migration, don't know, you could export the database and listen to binlog but that will lock table for a bit, depending on the database size. But maybe that's acceptable.

Would be curious to hear from folks with more DBA experience :)

Re: AWS Database Migration Service

#55
post #42

Earlier quoted context omitted.

Amazon probably wouldn't care if you did. They still get paid to shuffle the bits.

What they really want is keep you on RDS, not just get paid for the migration bandwidth.

Sure. But by forcing you to go through RDS even if you migrate X->RDS->Y, they are halfway there.

In the process you need to set up an AWS account, create a RDS database, import your data into it... and perhaps you stop there despite the initial plans. Or use RDS for your next project, since you are already more familiar with it.

Re: AWS Database Migration Service

#56
post #9

Earlier quoted context omitted.

Is it possible to run a read replica outside of RDS these days?

I don't see why not. Replication is just a command stream over a network. You can secure it over the daabsse's built in TLS support or over VPN. The only question is bandwidth and latency between your DC and AWS's.

I believe AWS stop you from connecting to anything outside your RDS subnet.

Re: AWS Database Migration Service

#57
post #9

Earlier quoted context omitted.

Is it possible to run a read replica outside of RDS these days?

Funny this showed up on here. I just spent the last 4-5 days playing with DMS. My company has been using AWS for a while, but management wanted a "backup" backup off Amazon "JUST IN CASE". Due to amazon blocking replication credentials for MYSQL servers we had to basically dump over scp to our off amazon server and run the update on that machine via a script. We tried a number of different options but none were relia…

Thanks for all the info. Is your On Amazon DB on RDS?

We use Postgres so it might be that we couldn't do the same. Presently I run the DB on EC2 instance(s) but one day I'm sure I'll switch to RDS. Just trying to understand what an exit strategy might look like in the future.

Re: AWS Database Migration Service

#58

Why is Amazon Redshift not a target for this service?

..or a source.

I've deployed a Redshift-based analytics application, and we're discovering operationally that -- for some workloads -- we would be better served by a Postgres RDS instance instead. It would be nice if I could one-click (or few-click) migrate everything from our Redshift instance to RDS.

Re: AWS Database Migration Service

#59
post #10
post #5

Looks like this could be a better way to migrate between major versions of PostgreSQL on RDS than rolling your own.

If you didn't catch it, AWS now supports upgrades between so-called "major versions" (eg. 9.3 -> 9.4). https://aws.amazon.com/about-aws/whats-new/2015/11/rds-postg...

I didn't catch that, thanks for point it out. Like another commenter said: this seems to be an in-place upgrade which is not advisable for major PostgreSQL versions. Additionally, it looks like this involves restarting the cluster (downtime). The Database Migration Service looks like it wouldn't require downtime.

Re: AWS Database Migration Service

#60

One of the biggest problems I've had with RDS (postgres specifically) is not being able to do incremental offsite backups. I wonder if the database migration service can help with that, as they say the process is "reliable" and "automatically restarts the process and continues the migration from where it was halted"

You can, you just need to implement it yourself in EC2 (other compute platforms are available) with trigger-based replication, using something like bucardo or londiste.

https://aws.amazon.com/blogs/aws/rds-postgres-read-replicas/

Post reply on HN