Live data from Hacker News

Computer Crash Wipes Out Years of Air Force Investigation Records

govexec.com

21–30 of 100 posts

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#21
post #10

Ah, the great flood of '76. Thousands of records were lost. Very convenient, that flood.

Genuine question - can you post some more information?

maybe the other poster meant this one? Oct 1976 Frederick, MD: http://www.fredericknewspost.com/archive/flood-flashback/art...

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#23
post #16
post #9

The joys of IT. Back in the day, to wipe years of records you had to burn or shred tons of paper in multiple locations. Now you just crash one little system... Despite 20 years of Microsoft-induced crashes, clear best practices and yadda yadda, most organizations will still have a single backup system for any given data repository -- if they have one at all. And the answer when shit happens is "outsource! Go to the c…

> The joys of IT. Back in the day, to wipe years of records you had to burn or shred tons of paper in multiple locations. Now you just crash one little system. [...] most organizations will still have a single backup system for any given data repository -- if they have one at all. No, back in the day, you could still wipe years of records because most people only used a single filing cabinet with no duplication or of…

16-18 million records destroyed in a fire in 1973:

http://www.archives.gov/st-louis/military-personnel/fire-197...

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#24
post #9

The joys of IT. Back in the day, to wipe years of records you had to burn or shred tons of paper in multiple locations. Now you just crash one little system... Despite 20 years of Microsoft-induced crashes, clear best practices and yadda yadda, most organizations will still have a single backup system for any given data repository -- if they have one at all. And the answer when shit happens is "outsource! Go to the c…

I would consider it progress. The only data loss to Google that I was able to find was [0], where a single datacenter was hit by lightning 4 times in short succession, and 0.000001% of disk space in said datacenter was lost. And only because the lost data consisted of recent writes that had yet to be backed up. And only because the lightning affected the AUX power systems in a way that they have since committed to fi…

IMO, cloud storage is the most reliable way to store data, and Google the most reliable

Does Google publish any durability numbers? Amazon claims 99.999999999% (11 9's) for S3 and Glacier (and considerably worse for EBS volumes). How does Google compare?

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#25
post #9

The joys of IT. Back in the day, to wipe years of records you had to burn or shred tons of paper in multiple locations. Now you just crash one little system... Despite 20 years of Microsoft-induced crashes, clear best practices and yadda yadda, most organizations will still have a single backup system for any given data repository -- if they have one at all. And the answer when shit happens is "outsource! Go to the c…

I would consider it progress. The only data loss to Google that I was able to find was [0], where a single datacenter was hit by lightning 4 times in short succession, and 0.000001% of disk space in said datacenter was lost. And only because the lost data consisted of recent writes that had yet to be backed up. And only because the lightning affected the AUX power systems in a way that they have since committed to fi…

The problem is not just DR, but anything that can result in data loss. There was the one where they'd delete stuff you didn't mean to [1], the one where they got stuff back by the scruff of the neck [2]... i can't be arsed to look further back, but I'm pretty sure shit happened earlier as well (and I'm just looking at GMail). And of course there are tons of horror stories about accounts disabled or deleted for commercial or other unspecified reasons, because once you put your trust in another entity, that's it, they can do as they please. Their track record might be as good as anyone's, but it still has a few holes and people should prepare for it rather than considering "the cloud" as a magic bucket.

(disclaimer: my life would probably be ruined by losing the gmail account I've held since private-beta days, because life is short and I'm shit at sysadmin; but there's a big difference between a lone old geek and large organizations...)

[1] http://www.dailymail.co.uk/sciencetech/article-2548010/Has-G...

[2] http://www.datacenterknowledge.com/archives/2011/03/01/googl...

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#26
There has been too much recently of government agencies losing data due to "crashes" or "inadvertent" deletions. For example the IRS, state department, and now the Air Force.

A really easy way to fix this is to make a law so that if government data such as this gets lost, then the cabinet level person responsible for overseeing the agency immediately loses their cabinet level position and is barred from further work with the federal government.

This would do wonders to align incentives all through the federal agencies when it comes to data safeguards.

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#28
post #10

Ah, the great flood of '76. Thousands of records were lost. Very convenient, that flood.

Genuine question - can you post some more information?

It was a quote from the TV show 'Yes, Minister':

James Hacker: How am I going to explain the missing documents to "The Mail"?

Sir Humphrey Appleby: Well, this is what we normally do in circumstances like these.

James Hacker: [reads memo] This file contains the complete set of papers, except for a number of secret documents, a few others which are part of still active files, some correspondence lost in the floods of 1967...

James Hacker: Was 1967 a particularly bad winter?

Sir Humphrey Appleby: No, a marvellous winter. We lost no end of embarrassing files.

James Hacker: [reads] Some records which went astray in the move to London and others when the War Office was incorporated in the Ministry of Defence, and the normal withdrawal of papers whose publication could give grounds for an action for libel or breach of confidence or cause embarrassment to friendly governments.

James Hacker: That's pretty comprehensive. How many does that normally leave for them to look at?

James Hacker: How many does it actually leave? About a hundred?... Fifty?... Ten?... Five?... Four?... Three?... Two?... One?... Zero?

Sir Humphrey Appleby: Yes, Minister.

(Source: http://www.imdb.com/character/ch0030014/quotes)

Edit: Although apparently a similar real-life incident occurred when Hurricane Sandy wiped out a significant portion of an FBI record archive: https://nsarchive.wordpress.com/2014/09/16/archival-neglect-...

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#29

There has been too much recently of government agencies losing data due to "crashes" or "inadvertent" deletions. For example the IRS, state department, and now the Air Force. A really easy way to fix this is to make a law so that if government data such as this gets lost, then the cabinet level person responsible for overseeing the agency immediately loses their cabinet level position and is barred from further work…

Yeah, forgive me if I'm too cynical, but Hanlon's razor fails when this sort of thing involves criminal investigations of government employees.

Edit: it also inspired a variation on an old joke about how the same word means different things to different people. For example, suppose the order goes out to "secure the data center."

Marines report back, "We have destroyed the data center."

Army reports, "We have killed everyone in the data center and are holding the position."

Navy: "We locked the doors when we left for the day."

Air Force: "We signed a three-year contract with an outsourcing company, with an option to extend at the same price for ten more years."

Which of course means that just because I don't think Hanlon's razor applies doesn't mean I think they're competent.

Re: Computer Crash Wipes Out Years of Air Force Investigation Records

#30

Dear 18F [1]: Help, please. [1] https://cloud.gov/

18F is severely outgunned. They need your help. I can't say that enough. The entire Western US is on fire, there's a drought, they have one fire hose and their next boss may turn it off.
Post reply on HN