Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

591–600 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#591
post #494
post #360

Earlier quoted context omitted.

As someone who’s fairly tech-literate but has a big blind spot in cryptography, I’d love to hear any suggestions you have for articles, blog posts, or smaller books on the topic! My (rudimentary, layman) understanding is that encryption is almost like a last line of defense and should never be assumed to be unbreakable. You sound both very knowledgeable on the topic, and very confident in the safety of modern encrypt…

Whew, that's actually a hard one! It's been long enough since I was getting into it that I'm not really sure what's the best present path on it. In terms of books, JP Aumasson's "Serious Cryptography" got a 2nd edition not too long ago and the first edition was good. Katz & Lindell's "Modern Cryptography" and Hoffstein's "Introduction to Mathematical Cryptography" are both standard texts that I think a lot of courses…

From someone else who was curious about an intelligent answer for the question in the comment above, thanks for taking the time to really deliver something interesting, politely too. Nice to see that not everyone here replies with arrogant disdain to someone who openly admits not knowing much about a complex field like cryptography, and nicely asking about it.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#592
post #536

Earlier quoted context omitted.

> The issue here is not refusing to use a foreign third party. That makes sense. Encrypt before sending to a third party?

Would you think that the U.S would encrypt gov data and store on Alibaba's Cloud? :)

Why not?

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#593

Article comments aside, it is entirely unclear to me whether or not there was no backups. Certainly no "external" backups, but potentially "internal" backups. My thinking is that not actually allowing backups and forcing all data there creates a prime target for the PRK folks right? I've been in low level national defense meetings about security where things like "you cannot backup off site" are discussed but there a…

They did have backups. But the backups were also destroyed in the same fire.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#594
post #183

Earlier quoted context omitted.

First of all, you cannot do much if you keep all the data encrypted on the cloud (basically just backing things up, and hope you don't have to fetch it given the egress cost). Also, availability is exactly the kind of issue that a fire cause…

Yeah backups would’ve been totally useless in this case. All South Korea could’ve done is restore their data from the backups and avoid data loss.

What part of the incident did you miss: the problem here was that they didn't backup in the first place.

You don't need the Cloud for backups, and there's no reason to believe that they would have backuped their data while using the cloud more than what they did with their self-hosting…

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#595
post #588

Earlier quoted context omitted.

Link? Am both curious and skeptical

Seagate Expansion drives are in this price range and can be shucked. They're not enterprise drives meant for constant operation, the big ones are Barracudas or maybe Exos, but for homelab NAS they're very popular.

I have such a NAS for 8 years, (and a smaller netgear one from maybe 16 years ago), and have yet such a disk fail. But you can get unlucky, buying a supposedly new but "refurbished" item via amazon or the seagate store (so I hear), or have the equivalent of the "death star" HDDs, which had a ridicilously high burnout rate (we measured something like > 10% of the drives failed every week across a fairly large deployhment in the field - major bummer.

If you use such consumer drives, I strongly suggest to make occasional offsite backups of large mostly static files (movies for most people I guess), and frequent backups of more volatile directories to an offsite place, maybe encrypted in the cloud.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#596
post #491

Goodness, I have over 100TB at home and it cost less than a two or three thousand dollars to put in place. That's like $25 per TB. > The stored data amounts to 858TB (terabytes), equivalent to 449.5 billion A4 sheets. No, the 858TB amounts to under $25k for the government of the 10th largest economy, of one of the most sophisticated countries on the planet, to put in place. Two of those would be less than the price o…

But it would not have been $25k, it would have been 1-2 million for an “enterprise grade” storage solution from Dell or a competitor. Which isn’t much compared with your granite mountain proposal, nor with the wages of 750,000 civil servants, but it’s a lot more than $25k.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#597

Earlier quoted context omitted.

> The government official who insisted that commercial AWS/GCP/Azure couldn't possibly be trusted with keeping the information They were still right though: it's absolutely clear without an ounce of doubt that whatever you put on an US cloud is being accessible by the US government, who can also decide to sanction you and deprive you from your ability to access the data yourself. Not having backups is entirely retard…

The U.S. Government can’t decrypt data for which it does not possess the key (assuming the encryption used is good).

Well first of all neither you and I knows the decryption capabilities of the NSA, all we know is that they have hired more cryptologists than the rest of the world combined.

Also, it's much easier for an intelligence service to get the hand on a 1kB encryption key than on a PB of data: the former is much easier to exfiltrate without being noticed.

And then I don't know why you bring encryption here: pretty much none of the use-case for using a cloud allow for fully encrypted data. (The only one that does is storing encrypted backups on the cloud, but the issue here is that the operator didn't do backups in the first place…)

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#598

The government official who insisted that commercial AWS/GCP/Azure couldn't possibly be trusted with keeping the information will be keeping their head low for a few days then... "The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for extern…

The issue here is not refusing to use a foreign third party. That makes sense. The issue is mandating the use of remote storage and not backing it up. That’s insane. It’s like the most basic amount of preparation you do. It’s recommended to even the smallest of companies specifically because a fire is a risk. That’s gross mismanagement.

Call me a conspiracy theorist, but this kind of mismanagement is intentional by design so powerful people can hide their dirty laundry.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#599

Earlier quoted context omitted.

My mind initially went to a government cover-up, but then: > 27th of September 2025, The fire is believed to have been caused while replacing Lithium-ion batteries. The batteries were manufactured by LG, the parent company of LG Uplus (the one that got hacked by the APT). Could the battery firmware have been sabotaged by the hacker to start the fire?

this was a plot in a Mr. Robot episode, heh. Life imitating art?

was there a battery hacking episode? I can't remember the show anymore, might be due in for a rewatch it seems.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#600
post #519
post #509

Earlier quoted context omitted.

The operational expenses of this stuff dwarfs the hardware cost. For the tape mountain, you need robots to confirm the tapes still work (mean time to detection of device failure and recovery are key for RAID durability computations). So, someone needs to constantly repair the robots or whatever. If I was being paid to manage that data set, I’d probably find two enterprise storage vendors, and stick two copies of the…

even with dual vendors, you'd have to still put in place a backup/restore procedures (with the associated software, which may need to be custom). Then you'd need regular testing. These operational concerns will basically double the cost yearly, probably.

You'll need permanent staff to oversee this, too. This will add at another ~500k+ to your annual expenditure.
Post reply on HN