Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About synchronous disk replication

cloud.google.com

Post reply on HN
Print view View original

1–10 of 28 posts

Re: About synchronous disk replication

#3

Post by gregwebs »

This would be great for a lot of purposes but still seems to fall short for operating a db in which you don’t want any data loss.

Although it is called synchronous replication, it appears to be asynchronous. That is the primary disk will acknowledge writes while the secondary may not have acknowledged them. To make something synchronous usually requires 2 secondaries where writes occur on 2/3 disks total: this allows one to fall behind and the primary acknowledges writes after replicating to 1/2 secondaries.

this GCP offering allows for monitoring when a secondary falls behind. With just one secondary that means when you get your alert, there’s a potential for data loss. If there was a write to 1/2 replicas then when you get the alert, you know you can still fail over to the secondary that is caught up and don’t have to panic while trying to deal with the replica that is falling behind.

Re: About synchronous disk replication

#7

Post by londons_explore »

I really don't know why Google doesn't just let users pick the how many copies to keep, and how many writes must be ACK'ed before the VM sees a write as complete.

Then users can decide the cost vs reliability vs durability of data written milliseconds before an outage.

Perhaps give users a web-based calculator where you can put the numbers, and see how much it would cost in $ per gb per day, the mean time to committed data loss based on historic data in years/centuries, and the typical increase in write latency (loss of performance) compared to a single replica.

Then the user can decide.

Re: About synchronous disk replication

#8

Post by kccqzy »

londons_explore wrote: post #7

I really don't know why Google doesn't just let users pick the how many copies to keep, and how many writes must be ACK'ed before the VM sees a write as complete. Then users can decide the cost vs reliability vs durability of data written milliseconds before an outage. Perhaps give users a web-based calculator where you can put the numbers, and see how much it would cost in $ per gb per day, the mean time to committe…

If you were a team working at Google internally, then yes you can pick these parameters. You can even pick between straight up replication versus Reed Solomon codes. I just don't understand why this is not exposed in their cloud offering.

Re: About synchronous disk replication

#9

Post by foota »

kccqzy wrote: post #8
londons_explore wrote: post #7

I really don't know why Google doesn't just let users pick the how many copies to keep, and how many writes must be ACK'ed before the VM sees a write as complete. Then users can decide the cost vs reliability vs durability of data written milliseconds before an outage. Perhaps give users a web-based calculator where you can put the numbers, and see how much it would cost in $ per gb per day, the mean time to committe…

If you were a team working at Google internally, then yes you can pick these parameters. You can even pick between straight up replication versus Reed Solomon codes. I just don't understand why this is not exposed in their cloud offering.

This is replication between zones, not within a single zone.

Re: About synchronous disk replication

#10

Post by londons_explore »

foota wrote: post #9
kccqzy wrote: post #8

Earlier quoted context omitted.

If you were a team working at Google internally, then yes you can pick these parameters. You can even pick between straight up replication versus Reed Solomon codes. I just don't understand why this is not exposed in their cloud offering.

This is replication between zones, not within a single zone.

The form would look like this:

    CREATE PERSISTENT DISK
    ----------------------

    Replicas:   W
    Zones to split replication across:  X
    How many replicas must complete a write to allow the VM to continue:  Y
    How many zones must complete a write to allow the VM to continue: Z

    With the above settings, you can expect approximately:

    Write latency 5-95%:   0.5-2.5 milliseconds
    Mean time to committed data loss:   37 years
    Mean time to failure to write:    3 years
    Mean time to data loss of data over 1 minute old:    18327 years.
    Cost:   $0.18/GB/month
The user would set those 4 parameters, with help text for guidance and a big red warning if you set the parameters to something insanely slow/unreliable.
Post reply on HN