Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google loses data as lightning strikes

bbc.com

Post reply on HN
Print view View original

111–120 of 142 posts

Re: Google loses data as lightning strikes

#111

Post by Dylan16807 »

mikeash wrote: post #77
idlewords wrote: post #24

Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…

Do they spell out what assumptions they're working under? Because, for example, I'm pretty sure that the odds of a civilization-ending asteroid or comet strike in the next year, while quite low, are higher than what's implied by 99.999999999% durability on trillions of objects."

Why do you specify trillions of objects? They practically promise data loss at that scale.

Also, the typical civilization-ending comet or asteroid is going to destroy 0 or 1 geographically-diverse data centers. Remember that this is a retention promise, not an uptime promise.

Re: Google loses data as lightning strikes

#112

Post by cjensen »

onethumb wrote: post #105
Tloewald wrote: post #68

Earlier quoted context omitted.

So empirically speaking their reliability is far, far worse than stated just based on that one data point.

How so? We'd need to know both how much data cjensen lost and how much data Glacier has in it to tell, neither of which I know from this conversation.

My recollection: around 20GB... however, they clearly were willing to try to get some of it back if that was helpful. It was just a redundant offsite backup for me, so there was no need.

Re: Google loses data as lightning strikes

#113

Post by bbcbasic »

Tloewald wrote: post #68
cjensen wrote: post #56

Earlier quoted context omitted.

Glacier is also 99.999999999%. But those nines are only true if Amazon's software is bug-free. I've lost data on Glacier on my home account. They personally called me to apologize.

So empirically speaking their reliability is far, far worse than stated just based on that one data point.

Not now they have fixed the bug.

Re: Google loses data as lightning strikes

#114

Post by MichaelGG »

TeMPOraL wrote: post #108
MichaelGG wrote: post #35

Earlier quoted context omitted.

Which would be embarrassing if this had happened to a product offering such an SLA. Instead, this is just zone-local HDD storage. It will only affect customers that decided to run a system in a single site, making no snapshots or using cross-region replication, etc. Which may be quite a few seeing as there seems to be this misunderstanding that VMs "in the cloud" have physics-defying redundancy attributes. (Not sayin…

Well, single VM is not "the cloud". It's just a VM on some dude's computer. Your clients seem to have been mislead by the incredibly content-free marketing around "the cloud".

Basically s/hosted/cloud-based is what everyone has done.

Re: Google loses data as lightning strikes

#116

Post by pacala »

wmf wrote: post #79
ai_ja_nai wrote: post #76

So Google has no redundancy at datacenter level?

Not for block storage (aka persistent disks): "we would like to take this opportunity to highlight an important reminder for our customers: GCE instances and Persistent Disks within a zone exist in a single Google datacenter and are therefore unavoidably vulnerable to datacenter-scale disasters. Customers who need maximum availability should be prepared to switch their operations to another GCE zone. For maximum dura…

Amazon builds datacenter redundancy in the same geographic locale. You can then setup synchronous replication between datacenters without atrocious latency, as all the disks are fairly close by, albeit powered [in an emergency] by independent generators.

OTOH, Google does datacenter redundancy across different locales, which make synchronous replication perform much worse, like you noted.

Re: Google loses data as lightning strikes

#117

Post by Aloha »

logicallee wrote: post #93
Aloha wrote: post #53

I work on cell sites, grounding system design and repair is a primary design element, even then, the presumption in the industry is that if a site takes a direct hit - or for that matter a nearby strike - the equipment is a total loss. The surge suppression gear we put in (lead ins at power feeds, RF feed, etc) is mostly to prevent a fire and to ensure the extra energy goes largely to ground.. but it won't prevent de…

Are you saying essentially that "There is no such thing as a surge protector, they don't physically exist. Only surge reducers exist." Because that's what it sounds like to me. EDIT: All right, I'll rephrase. According to Google's infobox from nat'l geographic, lightning generates up to 1 billion volts. -> Are surge protectors at even the highest-end data centers simply not rated to a billion volts of surge protectio…

Effectively yes, when you're on lightning scale, yes.

I use an isobar myself to power reduce noise, but I'm under no illusion that it will protect my shit from a direct strike.

Re: Google loses data as lightning strikes

#118

Post by nemo1618 »

danepowell wrote: post #94

This raises a relevant concern that's been on my mind: what's the best way to back up cloud services? Given that services like S3 and Google Drive have many more nines of durability than any local storage system I could devise, are backups even worth the trouble? There are a lot of cloud-to-cloud backup services out there, but to me that seems like the blind leading the blind, especially with regards to malicious dat…

If you really care about durability, your best best is erasure-coding + a wide geographic distribution of shards. For example, you could encode 1 TB of data into four shards, each shard containing 500 GB. You distribute these to servers in SF, NYC, Berlin, and Sydney. The key here is that you only need two shards to recover your 1 TB of data, and they can be any two shards. So if lightning strikes Berlin, and the Big One hits SF, your data is still safe. And thanks to erasure-coding, you can achieve this with only 2x redundancy (instead of 4x).
Post reply on HN