61–70 of 142 posts
This could potentially explain a lingering error on my Google Drive. Might there be movement within Google to contact the owners of data that was lost?
Earlier quoted context omitted.
This is S3 which isn't comparable to Google's persistent disks. S3 is equivalent to Google Cloud Storage which has "99.999999999%" durability as per https://cloud.google.com/storage/ To accurately compare them you'd need to look at AWS EBS: "Amazon EBS volumes are designed for an annual failure rate (AFR) of between 0.1% - 0.2%, where failure refers to a complete or partial loss of the volume, depending on the size a…
The point isn't to compare competing cloud services, but draw attention to the fantastical nature of these durability promises.
" This outage is wholly Google's responsibility. However, we would like to take this opportunity to highlight an important reminder for our customers: GCE instances and Persistent Disks within a zone exist in a single Google datacenter and are therefore unavoidably vulnerable to datacenter-scale disasters. Customers who need maximum availability should be prepared to switch their operations to another GCE zone. For maximum durability we recommend GCE snapshots and Google Cloud Storage as resilient, geographically replicated repositories for your data. "
In a cloud based infra, one still needs to know the difference. And for the ultra-low numbers, they are from mathematics I think. (# replicas, possibly cross DC/geo)
I use google drive a lot, I don't track what's in my drive, should I be worried?
Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
(Forgetting about correlation was a big part of the MBS and LTCM financial failures)
Earlier quoted context omitted.
Nowadays they use Reed–Solomon coding to effectively distribute their data without copying it to 3 places.
Do you have a source for that? Why would they start using it now?
link here: http://static.googleusercontent.com/external_content/untrust...
actually in colossus one can tune RS coding parameters per file, to get a tradeoff between performance/durablity.
RS coding uses less copies, but same level of safety (tradeoff is the recovery computation time.)
Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
I wouldn't trust any of these figures unless they have ongoing efforts to test them empirically. E.g. create distributed databases of 100 trillion objects, mess with them in various ways, and perform correctness checks on them.
Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
At this point, correlation matters. In the unlikely event that I lose at least 1 object, what is the conditional probability of losing another? (Forgetting about correlation was a big part of the MBS and LTCM financial failures)
Assumptions that variables are independent are often very mistaken.
Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
Glacier is also 99.999999999%. But those nines are only true if Amazon's software is bug-free. I've lost data on Glacier on my home account. They personally called me to apologize.
Earlier quoted context omitted.
At this point, correlation matters. In the unlikely event that I lose at least 1 object, what is the conditional probability of losing another? (Forgetting about correlation was a big part of the MBS and LTCM financial failures)
And Fukushima. Assumptions that variables are independent are often very mistaken.