"Lightning crashes, an old server dies...."
Oh, I see you're Selling the Drama ...
Google loses data as lightning strikes
51–60 of 142 posts
Re: Google loses data as lightning strikes
#52Thats why you should always write it down folks...
Re: Google loses data as lightning strikes
#53I work on cell sites, grounding system design and repair is a primary design element, even then, the presumption in the industry is that if a site takes a direct hit - or for that matter a nearby strike - the equipment is a total loss.
The surge suppression gear we put in (lead ins at power feeds, RF feed, etc) is mostly to prevent a fire and to ensure the extra energy goes largely to ground.. but it won't prevent dead gear.
Re: Google loses data as lightning strikes
#54Wow. NSA attacks are really leveling up.
Re: Google loses data as lightning strikes
#55Earlier quoted context omitted.
[deleted]
AFAIK google doesn't use hardware-based arrays in their servers. I think the reference to batteries would more likely be in reference to a DRUPS (Diesel Rotary UPS) which most datacenters run, sometimes with some form of battery in combination with the flywheel. Typically the combination of kinetic energy in the flywheel (and potentially batteries) only hold enough power to last for 30 seconds or so (often as low as…
From http://www.cnet.com/news/google-uncloaks-once-secret-server-...
"Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity"
Re: Google loses data as lightning strikes
#56Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
Glacier is also 99.999999999%.
But those nines are only true if Amazon's software is bug-free. I've lost data on Glacier on my home account. They personally called me to apologize.
Re: Google loses data as lightning strikes
#57Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
Many people have far more than 10k objects though. Based on that math, if you stored 10 billion objects in S3 (certainly well within the realm of possibility), you'd lose an object on average every 10 years, which is a length of time one can think about. Your main point (that making high reliability systems more conventionally reliable is building your high wall yet higher still) is definitely valid. But the lossage…
I completely agree with you. Where they are intentionally misleading is in pretending that keeping one object for a zillion years is the same as keeping a zillion objects for one year.
Re: Google loses data as lightning strikes
#58Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
Indeed "on average" covers the entire period and from initial local-node intake of the data it has to then be propergated to another site to increase integrity probability. So if it takes even 5 seconds to propergate to several sites at different locations then the durability scales from when the clock starts.
Heck once in every 10,000,000 years, the odds of winning the USA lottery are around one in 175,000,000 and yet people partake and somebody wins. So it is always worth looking at odds from another perspective and accept it is either impossible, probably or possible and as we know the return from investment in durability is logarithmic in cost going up and return getting smaller.
Still one of the biggest issues in any datacentre is not just the power, but also the quality of the power and even the slightest noise could and does increase the odds of some electronics going wrong. UPS's (though been many years) will put out a square wave in modulation and the ideal is a perfect sine-wave. It is truly educational to see the quality of many power sources on an oscilloscope. Also equipment on the same circuit can also induce noise, so can vary rack to rack.
Re: Google loses data as lightning strikes
#59Wouldn't successive lightning strikes be more likely due to ionisation of the air?
Well, there's the opposite argument:
"Wouldn't successive lightning strikes be less likely due to reduced local electric potential?"
It's not readily apparent to me which of the effects will be larger.
Re: Google loses data as lightning strikes
#60Relevant amusing bit from the Amazon FAQ: "S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years." I think my favorite part of that is "on average", a…
The irony is that, by that time, Amazon will almost certainly not be around. So doing it in terms of time is actually a different calculation, and this is based on wrong assumptions :)