Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google loses data as lightning strikes

bbc.com

Post reply on HN
Print view View original

71–80 of 142 posts

Re: Google loses data as lightning strikes

#71

Post by decker »

chatmasta wrote: post #18

> Google said that just 0.000001% of disk space was permanently affected. Assuming 1 petabyte of total storage at the datacenter, that equates to about 100mb. I wonder how much storage they have there.

I think your estimate is too low by 1000x. 1PB is 500 2T drives. There's no way Google would run a data center with only 25-250 hosts.

Re: Google loses data as lightning strikes

#73

Post by bentpins »

decker wrote: post #71
chatmasta wrote: post #18

> Google said that just 0.000001% of disk space was permanently affected. Assuming 1 petabyte of total storage at the datacenter, that equates to about 100mb. I wonder how much storage they have there.

I think your estimate is too low by 1000x. 1PB is 500 2T drives. There's no way Google would run a data center with only 25-250 hosts.

https://what-if.xkcd.com/63/

XKCD guessed 15EB in 2013

Re: Google loses data as lightning strikes

#74

Post by bskap »

djhworld wrote: post #63

Do they know what customers were affected? I use google drive a lot, I don't track what's in my drive, should I be worried?

I think it only impacted people who were using Compute Engine and storing data on a hard drive instead of using Cloud Storage. I suspect that Drive data has redundant copies stored in multiple data centers in addition to frequent backups.

Re: Google loses data as lightning strikes

#75

Post by tedchs »

Key point from Google's status incident page that clarifies some of the partially-incorrect statements in the press and comments about this:

"In a very small fraction of cases (less than 0.000001% of PD space in europe-west1-b), there was permanent data loss."

https://status.cloud.google.com/incident/compute/15056#57195...

Re: Google loses data as lightning strikes

#77

Post by mikeash »

idlewords wrote: post #24

Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…

Do they spell out what assumptions they're working under? Because, for example, I'm pretty sure that the odds of a civilization-ending asteroid or comet strike in the next year, while quite low, are higher than what's implied by 99.999999999% durability on trillions of objects."

Re: Google loses data as lightning strikes

#78

Post by keithpeter »

chinathrow wrote: post #72
impostervt wrote: post #16

Curious - how do they know lightning hit four times? Was someone outside counting?

Yes, lots of folks are counting. http://www.blitzortung.org/

Thanks very much for posting this link. Could be really handy for physics teaching next year.

Re: Google loses data as lightning strikes

#79

Post by wmf »

ai_ja_nai wrote: post #76

So Google has no redundancy at datacenter level?

Not for block storage (aka persistent disks): "we would like to take this opportunity to highlight an important reminder for our customers: GCE instances and Persistent Disks within a zone exist in a single Google datacenter and are therefore unavoidably vulnerable to datacenter-scale disasters. Customers who need maximum availability should be prepared to switch their operations to another GCE zone. For maximum durability we recommend GCE snapshots and Google Cloud Storage as resilient, geographically replicated repositories for your data."

Achieving RPO=0 generally requires synchronous replication to a different datacenter which adds significant latency.

Re: Google loses data as lightning strikes

#80

Post by devindotcom »

idlewords wrote: post #57
emmett wrote: post #37

Earlier quoted context omitted.

Many people have far more than 10k objects though. Based on that math, if you stored 10 billion objects in S3 (certainly well within the realm of possibility), you'd lose an object on average every 10 years, which is a length of time one can think about. Your main point (that making high reliability systems more conventionally reliable is building your high wall yet higher still) is definitely valid. But the lossage…

I completely agree with you. Where they are intentionally misleading is in pretending that keeping one object for a zillion years is the same as keeping a zillion objects for one year.

Keeping one object for a zillion years sounds like an interesting project, now that you bring it up.

Our data integrity is such that it will survive universal cataclysm and the reduction of all data to a single point of infinite energy.[1]

[1]guarantee applies to a single bit of data[2] [2]bit value must be 1

Post reply on HN