Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

Post reply on HN
Print view View original

711–720 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#711

Post by trivo »

0xDEAFBEAD wrote: post #640
sbinnee wrote: post #555

What a sad news as a Korean to see a post about Korea at the top of HN during one of the largest Korean holiday. I can share an anecdote how slow tech adoption is in Korea. It is not exactly about tech in public section but in private companies. I assume public section has slower adoption rate than private ones in general. Just about a year ago I had a couple of projects with insurance companies. I won't name them bu…

Things South Korea is good at producing: Cars, ships, steel, semiconductors, electronics, medicines, tanks, aircraft parts, nuclear reactors... Things South Korea is bad at producing: Software. Not too bad overall.

Seems everyone ouside of US is bad at producing software.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#712

Post by mliezun »

quantumsequoia wrote: post #327
nowittyusername wrote: post #314

I must say, at least for me personally when I hear about such levels of incompetence it rings alarm bells in my head making me think that maybe intentional malice was involved. Like someone higher up had set up the whole thing to happen in such a matter because there was a benefit to this happening we are unaware of. I think this belief maybe stems from lack of imagination on how really stupid humans can get.

Most people overestimate the prevalence of malice, und underestimate the prevalence of incompetence

What do you make of this? The guy who was in charge of restoring the system was found dead

https://www.thestar.com.my/aseanplus/aseanplus-news/2025/10/...

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#713

Post by deaux »

alephnerd wrote: post #219
neom wrote: post #131

Earlier quoted context omitted.

Good thing Korea has cloud providers, apparently Kakao has even gone...beyond the cloud! https://kakaocloud.com/ https://www.nhncloud.com/ https://cloud.kt.com/ To name a few.

They are overwhelmingly whitelabeled providers. For example, Samsung SDI Cloud (the largest "Korean" cloud) is an AWS white label. Korea is great at a lot of engineering disciplines. Sadly, software is not one of them, though it's slowly changing. There was a similar issue a couple years ago where the government's internal intranet was down a couple days because someone deployed a switch in front of outbound connecti…

That doesn't seem accurate at all. The big 3 Korean clouds used inside Korea are NHN Cloud, Naver Cloud and now KT. Which one of these is whitelabeled? And what's the source on Samsung SDI Cloud being the "largest Korean cloud"? What metric?

NHN Cloud is in fact being used more and more in the government [1], as well as playing a big part in the recovery effort of this fire. [2]

No, unlike what you're suggesting, Korea has plenty of independent domestic cloud and the government has been adopting it more and more. It's not on the level of China, Russia or obviously the US, but it's very much there and accelerating quickly. Incomparable to places like the EU which still have almost nothing.

[1] https://www.ajunews.com/view/20221017140755363 - 2022, will have grown a lot now [2] https://www.mt.co.kr/policy/2025/10/01/2025100110371768374

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#714

Post by southernplaces7 »

throwaway290 wrote: post #665
southernplaces7 wrote: post #564

Earlier quoted context omitted.

Not the same country but another example of a culturally similar attitude towards shame over failure: In Japan in 1985, Flight 123, a massive Boeing 747 carrying 524 people, lost control shortly after takeoff from Tokyo en route to Osaka. The plane's aft pressure bulkhead catastrophically exploded, causing total decompression at the high altitude, severing all four of the massive plane's hydraulic stabilizer systems…

Happened more recently too https://www.straitstimes.com/asia/east-asia/south-korean-ex-...

Interesting case too, and that he committed suicide despite not really being blamed from what I just read.

On the other hand you have cases like the MV Wewol ferry disaster, also in South Korea, in which well over 250 passengers died horribly. Most of them were just kids, high school students on a trip. The causes leading up to the tragedy, the accident management by the crew itself and the subsequent rescue, body retrieval and investigation, were absolutely riddled with negligence, incompetence, bad management and all kinds of blame shifting.

The owner of the ferry company itself had an arrest warrant issued for him, then fled and only later was found in a field dead and presumed to have committed suicide.

Underlying all this is that even these apparent cultural ideas of committing suicide to atone for the shame of some gigantic mistake don't seem to prevent people from actually making these kinds of mistakes or doing things more responsibly in the first place.

https://en.wikipedia.org/wiki/Sinking_of_MV_Sewol

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#715

Post by antonvs »

alsetmusic wrote: post #485
Marsymars wrote: post #395

Earlier quoted context omitted.

This is how I’ve done translation for a number of years, even pre-LLM, between the languages I speak natively - machine translation is good enough that it’s faster for me to fix its problems than for me to do it from scratch. (Whether machine translation uses LLMs or not doesn’t seem especially relevant to the workflow.)

My partner is a pro-democracy fighter for her country of origin (she went to prison for it). She used to translate english articles of interest to her native language for all the fellow-exiles from her country. I showed her Google translate and it blew her mind how much work it did for her. All she had to do was review it and clean it up. The AI hype train is bs, but there're real and concrete uses for it if you don'…

> The AI hype train is bs, but there're real and concrete uses for it

When you consider that there are real and concrete uses for it across a wide variety of domains, the hype starts to make more sense.

Obviously Sam “we’ll build a Dyson sphere with it” is off in hype lala land somewhere while he tries to raise a trillion dollars to burn through fossil fuels as fast as possible, but that’s a kind of symptom of the real underlying capabilities and promise here.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#716

Post by throwaway_fjmr »

egorfine wrote: post #697
hiharryhere wrote: post #687

Earlier quoted context omitted.

Government isn’t perfect but I’d be interested to know what alternative you propose?

a) Incarceration time for IT execs and responsible engineers. b) Let companies go out of business once they fail to protect their own crucial data. None of that is possible.

so you are not proposing anything real then? I can pull "magic indestructible backup solution" out of my arse, too :(

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#718

Post by vbezhenar »

hapanin wrote: post #409
vbezhenar wrote: post #320

Earlier quoted context omitted.

S3 does not spend 3x drives to provide redundancy. Probably 20% more drives or something like that. They split data to chunks and use erasure coding to store them in multiple drives with little overhead.

wait, can you elaborate on how this works?

You have 100 bytes file. You split it into 10 chunks (data shards) and add 11-th chunk (parity shard) as XOR of all 10 chunks. Now you store every chunk on separate drive. So you have 100 bytes and you spent 110 bytes to store them all. Now you can survive one drive death, because you can recompute any missing chunk as XOR of all chunks.

That's very primitive explanation, but should be easy to understand.

In reality S3 uses different algorithm (probably Reed-Solomon codes) and some undisclosed number of shards (probably different for different storage classes). Some say that they use 5 of 9 (so 5 data shards + 4 parity shards which makes for 80% overhead), but I don't think it's official information.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#719

Post by Ylpertnodi »

ptojr wrote: post #619

I wouldn't be surprised if someone caused this intentionally.

> I wouldn't be surprised if someone caused this intentionally.

What, no backup(s) set up? Hmmm, possibly. But, they're'd be a paper trail.

Imagine all the scrabbling going on right now - people desperately starting to cover their arses. But chances are, what they need has just burnt down, with no backups.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#720

Post by belorn »

efitz wrote: post #537
belorn wrote: post #404

Earlier quoted context omitted.

As a sysadmin at company that provide fairly sensitive services, I find online cloud backups to be way to slow for the purpose of protecting against something like the server room being destroyed by a fire. Even something like spinning disks at a remote location feel like a risk, as files would need to be copied onto faster disks before services could be restored, and that copying would take precious time during an e…

For truly colossal amounts of data, fedex has more bandwidth than fiber. I don’t know if any cloud providers will send you your stuff on physical storage, but most will allow you to send your stuff to them on physical storage- eg AWS snowball. There are two main reasons why people struggle with cloud restore: 1. Not enough incoming bandwidth. The cloud’s pipe is almost certainly big enough to send your data to you. Y…

If you allow it to cost a bit, which is likely a good choice given the problem, then there are several solutions available. It is important to think through the scenario, and if possible, do a dry run of the solution. A remote physical server can work quite well and be cost effective compared to a flagship storage tier, and if data security is important, you can access the files on your own server directly rather than downloading an encrypted blob from a cloud located outside the country.
Post reply on HN