Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

631–640 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#631
post #382
post #122

Earlier quoted context omitted.

Woah, read the timeline at the top of this. The fire happened the very day the government ordered onsite inspection was supposed to start due to Chinese/NK hacking.

Such coincidences do happen. 20 years ago the plane which was carrying all the top brass of the Russian Black Sea Fleet as well as the Fleet’s accounting documentation for inspection to Moscow burst in flames and fell to the ground while trying to get airborne. Being loaded with fuel it immediately became one large infernal fireball. By some miracle no top brass suffered even minor burn/injury while all the accountin…

Quite a few of those top brass years later shot themselves in the head several times before jumping from a window.

Anyway, shoe production has never been better.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#632

The easy solution would be to use something like Amazon S3 to store documents as objects and let them worry about backup; but governments are worried (and rightly so) about the US government spying on them. Thus, the not-so-easy-but-arguably-better solution would be to self-host an open source S3-compatible object storage solution. Are there any good open source alternatives to S3?

[dead]

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#633

The easy solution would be to use something like Amazon S3 to store documents as objects and let them worry about backup; but governments are worried (and rightly so) about the US government spying on them. Thus, the not-so-easy-but-arguably-better solution would be to self-host an open source S3-compatible object storage solution. Are there any good open source alternatives to S3?

I recently learned about https://garagehq.deuxfleurs.fr/ but i have no expirience using it

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#634

Earlier quoted context omitted.

The issue here is not refusing to use a foreign third party. That makes sense. The issue is mandating the use of remote storage and not backing it up. That’s insane. It’s like the most basic amount of preparation you do. It’s recommended to even the smallest of companies specifically because a fire is a risk. That’s gross mismanagement.

Backups should be far away, too. Apparently some companies lost everything on 9/11 because their backups were in the other tower.

[dead]

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#635

Earlier quoted context omitted.

Call me a conspiracy theorist, but this kind of mismanagement is intentional by design so powerful people can hide their dirty laundry.

Never attribute to malice what can be attributed to stupidity. There was that time when some high profile company's entire Google Cloud account was destroyed. Backups were on Google Cloud too. No off-site backups.

One of the data integrity people sadly committed suicide as a result of this fire, so I am also thinking this was an incompetence situation (https://www.yna.co.kr/view/AKR20251003030351530).

For the budget spent, you’d think they would clone the setup in Busan and sync it daily or something like this in lieu of whatever crazy backup they needed to engineer but couldn’t.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#636

Earlier quoted context omitted.

Call me a conspiracy theorist, but this kind of mismanagement is intentional by design so powerful people can hide their dirty laundry.

Never attribute to malice what can be attributed to stupidity. There was that time when some high profile company's entire Google Cloud account was destroyed. Backups were on Google Cloud too. No off-site backups.

You have to balance that with how low can you expect human beings to lower their standards when faced with bureaucratic opposition. No backups on a key system would increase the likelihood of malice versus stupidity, since the importance of backups is known to IT staff regardless of role and seniority for only 40 years or so.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#637

Earlier quoted context omitted.

Call me a conspiracy theorist, but this kind of mismanagement is intentional by design so powerful people can hide their dirty laundry.

Never attribute to malice what can be attributed to stupidity. There was that time when some high profile company's entire Google Cloud account was destroyed. Backups were on Google Cloud too. No off-site backups.

You were probably thinking of UniSuper [0], an Australian investment company with more than $80B AUM.

Their 3rd party backups with another provider were crucial to helping them undo the damage from the accidental deletion by GCloud.

GCloud eventually shared a post-mortem [1] about what went down.

0: https://news.ycombinator.com/item?id=40304666

1: https://cloud.google.com/blog/products/infrastructure/detail...

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#638
post #13

Earlier quoted context omitted.

It's even worse. According to other articles [1], the total data of "G drive" was 858 TB. It's almost farcical to calculate, but AWS S3 has pricing of about $0.023/GB/month, which means the South Korean government could have reliable multi-storage backup of the whole data at about $20k/month. Or about $900/month if they opted for "Glacier deep archive" tier ($0.00099/GB/month). They did have backup of the data ... in…

AWS? Linus Tech Tips has run multiple petabyte servers in their server closet just for sponsor money and for the cool of it. No need to outsource your national infrastructure to foreign governments, a moderate (in government terms) investment in a few racks across the country could've replicated everything for maybe half a year's worth of Amazon subscription fees.

Exactly, everyone here on hackernews is talking about Azure/AWS/GCP as if it was the only correct way to store data. Americans are too self centered, it's quite crazy.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#640

What a sad news as a Korean to see a post about Korea at the top of HN during one of the largest Korean holiday. I can share an anecdote how slow tech adoption is in Korea. It is not exactly about tech in public section but in private companies. I assume public section has slower adoption rate than private ones in general. Just about a year ago I had a couple of projects with insurance companies. I won't name them bu…

Things South Korea is good at producing: Cars, ships, steel, semiconductors, electronics, medicines, tanks, aircraft parts, nuclear reactors...

Things South Korea is bad at producing: Software.

Not too bad overall.

Post reply on HN