Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

Post reply on HN
Print view View original

331–340 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#331

Post by n5NOJwkc7kRC »

hinkley wrote: post #202
CamouflagedKiwi wrote: post #160

Earlier quoted context omitted.

And yet here is an example where keeping critical data off public cloud storage has been significantly worse for them in the short term. Not that they should just go all in on it, but an encrypted copy on S3 or GCS would seem really useful right about now.

We’ve had Byzantine crypto key solutions since at least 2007 when I was evaluating one for code signing for commercial airplanes. You could put an access key on k:n smart cards, so that you could extract it from one piece of hardware to put on another, or you could put the actual key on the cards so burning down the data center only lost you the key if you locked half the card holders in before setting it on fire.

SSS is from 1979. https://en.wikipedia.org/wiki/Shamir%27s_secret_sharing

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#332

Post by ahartmetz »

skissane wrote: post #286
alephnerd wrote: post #219

Earlier quoted context omitted.

They are overwhelmingly whitelabeled providers. For example, Samsung SDI Cloud (the largest "Korean" cloud) is an AWS white label. Korea is great at a lot of engineering disciplines. Sadly, software is not one of them, though it's slowly changing. There was a similar issue a couple years ago where the government's internal intranet was down a couple days because someone deployed a switch in front of outbound connecti…

I spent a week of my life at a major insurance company in Seoul once, and the military style security, the obsession with corporate espionage, when all they were working on was an internal corporate portal for an insurance company… The developers had to use machines with no Internet access, I wasn’t allowed to bring my laptop with me lest I use it to steal their precious code. A South Korean colleague told me it was…

I've done some work for a large SK company and the security was manageable. Certainly higher than anything I've seen before or after and with security theater aspects, but ultimately it didn't seriously get in the way of getting work done.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#333

Post by lucb1e »

rvba wrote: post #119
yongjik wrote: post #13

Earlier quoted context omitted.

It's even worse. According to other articles [1], the total data of "G drive" was 858 TB. It's almost farcical to calculate, but AWS S3 has pricing of about $0.023/GB/month, which means the South Korean government could have reliable multi-storage backup of the whole data at about $20k/month. Or about $900/month if they opted for "Glacier deep archive" tier ($0.00099/GB/month). They did have backup of the data ... in…

How does this even make sense business wise for AWS? Is their cost per unit so low?

It's expensive if you calculate what it would cost for a third party to compete with. Or see e.g. this graph from a recent HN submission: https://si.inc/posts/the-heap/#the-cost-breakdown-cloud-alte...

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#334

Post by vgivanovic »

alephnerd wrote: post #219
neom wrote: post #131

Earlier quoted context omitted.

Good thing Korea has cloud providers, apparently Kakao has even gone...beyond the cloud! https://kakaocloud.com/ https://www.nhncloud.com/ https://cloud.kt.com/ To name a few.

They are overwhelmingly whitelabeled providers. For example, Samsung SDI Cloud (the largest "Korean" cloud) is an AWS white label. Korea is great at a lot of engineering disciplines. Sadly, software is not one of them, though it's slowly changing. There was a similar issue a couple years ago where the government's internal intranet was down a couple days because someone deployed a switch in front of outbound connecti…

I am very happy with the software that powers my Hyundai Tuscon hybrid. (It's a massive system that runs the gas and electric engines, recharging, shifting gears, braking, object detection, and a host of information and entertainment systems.) After 2 years, 0 crashes and no observable errors. Of course, nothing is perfect: maps suck. The navigation is fine; it's the display that is at least 2 decades behind the times.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#335

Post by john-tells-all »

This is literally comic. The plot of the live action comic book movie "Danger: Diabolik" [1] has a segment where the a country's tax records are destroyed, thus making it impossible for the government to collect taxes from its citizens.

[1] https://en.wikipedia.org/wiki/Danger:_Diabolik

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#336

Post by jml78 »

MangoCoffee wrote: post #76

It's hard to believe this happened. South Korea has tech giants like Samsung, and yet this is how the government runs? Is the US government any better?

Yes. The US government requires offsite backups .

They also require routine testing distaster recovery plans.

I participated in so many different programs over the years with those tests.

Tests that would roll over to facilities across the country

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#337

Post by xoa »

jacquesm wrote: post #282
JumpCrisscross wrote: post #274

Earlier quoted context omitted.

> the should not have kept that data on foreign cloud storage regardless. It's not like there are only two choices here Doesn't have to be an American provider (Though anyone else probably increases Seoul's security cross section. America is already its security guarantor, with tens of thousands of troops stationed in Korea.) And doesn't have to be permanent. Ship encrypted copies to S3 while you get your hardenede-b…

I'm aware of a big cloud services provider (I won't name any names but it was IBM) that lost a fairly large amount of data. Permanently. So that too isn't a guarantee. They simply should have made local and off-line backups, that's the gold standard, and to ensure that those backups are complete and can be used to restore from scratch to a complete working service.

>I'm aware of a big cloud services provider (I won't name any names but it was IBM) that lost a fairly large amount of data. Permanently. So that too isn't a guarantee.

Permanently losing data at a given store point isn't relevant to losing data overall. Data store failures are assumed or else there'd be no point in backups. What matters is whether failures in multiple points happen at the same time, which means a major issue is whether "independent" repositories are actually truly independent or whether (and to what extent) they have some degree of correlation. Using one or more completely unique systems done by someone else entirely is a pretty darn good way to bury accidental correlations with your own stuff, including human factors like the same tech people making the same sorts of mistakes or reusing the same components (software, hardware or both). For government that also includes political factors (like any push towards using purely domestic components).

>They simply should have made local and off-line backups

FWIW there's no "simply" about that though at large scale. I'm not saying it's undoable at all but it's not trivial. As is literally the subject here.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#338

Post by senko »

vbezhenar wrote: post #313
lukevp wrote: post #310

Earlier quoted context omitted.

How’s that? Using encryption, which is known to have backdoors and is vulnerable to nation state cracking?

It is much more likely and cheaper, that US marines will desant and capture your backup facility, than someone would break AES-128.

Sending troops would be an act of war, and definitely not cheap.

Stealing some encryption keys, just another Wednesday.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#339

Post by charlieyu1 »

eCa wrote: post #120
kristianc wrote: post #95

The government official who insisted that commercial AWS/GCP/Azure couldn't possibly be trusted with keeping the information will be keeping their head low for a few days then... "The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for extern…

Agree completely that it's absolute wild to run such a system without backups. But at this point no government should keep critical data on foreign cloud storage.

You don’t need cloud when you have the data centre, just backups in physical locations somewhere else
Post reply on HN