Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

Post reply on HN
Print view View original

761–770 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#761

Post by peteforde »

hnlmorg wrote: post #738
osivertsson wrote: post #700

Earlier quoted context omitted.

Laws of physics hasn't changed since the early 00s though, we could build very low latency point to point links back then too.

Switching gear was slower and laying new fibre wasn't an option for your average company. Particularly not point-to-point between your DB server and your replica. So if real-time synchronization isn't practical, you are then left to do out-of-hours backups and there you start running into bandwidth issues of the time.

Never underestimate the potential packet loss of a Concorde filled with DVDs.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#762

Post by krageon »

Kinrany wrote: post #662
jeena wrote: post #511

Earlier quoted context omitted.

I've been working for a Korean Hyundai supplier for two years training them in modern software development processes. The programming part is not a problem, they have a lot of talented people. The big problem from my point of view is management. Everyone pushes responsibility and work all the way down to the developera so that they do basically everything themselves from negotiating with the customer, writing the req…

> If they're late,they just stay and work longer and on the weekends and sleep at the desk. This is the only part that sounds bad? Negotiating with customers may require some help as well but it's better than having many layers in between.

If the dev does everything, their manager may as well be put in a basket and pushed down the river. You can be certain there are a lot of managers. The entire storyline sounds like enterprise illness to me to be honest.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#763

Post by Cthulhu_ »

sneak wrote: post #502
eCa wrote: post #120

Earlier quoted context omitted.

Agree completely that it's absolute wild to run such a system without backups. But at this point no government should keep critical data on foreign cloud storage.

It's 2025. Encryption is a thing now. You can store anything you want on foreign cloud storage. I'd give my backups to the FSB.

There's certifications too, which you don't get unless you conform to for example EU data protection laws. On paper anyway. But these have opened up Amazon and Azure to e.g. Dutch government agencies, the tax office will be migrating to Office365 for example.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#764

Post by Cthulhu_ »

vladms wrote: post #170
CamouflagedKiwi wrote: post #160

Earlier quoted context omitted.

And yet here is an example where keeping critical data off public cloud storage has been significantly worse for them in the short term. Not that they should just go all in on it, but an encrypted copy on S3 or GCS would seem really useful right about now.

You can do a bad job with public or private cloud. What if they would have had the backup and lost the encryption key? Cost wise probably having even a Korean different data center backup would not have been huge effort, but not doing it exposed them to a huge risk.

Then they didn't have a correct backup to begin with; for high profile organizations like that, they need to practice outages and data recovery as routine.

...in an ideal world anyway, in practice I've never seen a disaster recovery training. I've had fire drills plenty of times though.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#765

Post by Cthulhu_ »

63stack wrote: post #255
jn78 wrote: post #80

https://phrack.org/issues/72/7_md#article

This is the first time I see this site, who/what is phrack? A hacker group?

It looks delightful, but definitely for and by a specific subculture.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#766

Post by Cthulhu_ »

phatfish wrote: post #736

"The stored data amounts to 858TB (terabytes), equivalent to 449.5 billion A4 sheets" Just so we can all visualise this in an understandable way, if laid end-to-end how many times round the world would the A4 sheets go? And what is their total area in football fields?

I love that people are still trying to put data on A4s and we're long past the point of being able to visualize it.

That said, if I'm ever fuck-you rich, I'm going to have a pyramid built to bury me in and a library of hardcover printed wikipedia.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#767

Post by flakeoil »

trivo wrote: post #711
0xDEAFBEAD wrote: post #640

Earlier quoted context omitted.

Things South Korea is good at producing: Cars, ships, steel, semiconductors, electronics, medicines, tanks, aircraft parts, nuclear reactors... Things South Korea is bad at producing: Software. Not too bad overall.

Seems everyone ouside of US is bad at producing software.

Yes, and US bad at everything else.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#768

Post by lambdaone »

SEJeff wrote: post #737
flumpcakes wrote: post #673

Earlier quoted context omitted.

Companies big enough will lay the fibre. 50-100 miles of fibre isn't much if you are a billion dollar business. Even companies like BlackRock who had their own datacenters have since taken up Azure. 50 miles latency is negligible, even for databases.

In the US, dark fiber will run you around 100k / mile. Thats expensive for anyone even if they can afford it. I worked in HFT for 15 years and we had tons of it.

DWDM per-wavelength costs are way, way lower than that, and, with the optional addition of encryption, perfectly secure and fast enough for disk replication for most storage farms. I've been there and done it.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#769

Post by veeti »

southernplaces7 wrote: post #564
covercash wrote: post #302

Earlier quoted context omitted.

If the US government and corporate executives had even half this level of shame, we'd have nobody left in those positions!

Not the same country but another example of a culturally similar attitude towards shame over failure: In Japan in 1985, Flight 123, a massive Boeing 747 carrying 524 people, lost control shortly after takeoff from Tokyo en route to Osaka. The plane's aft pressure bulkhead catastrophically exploded, causing total decompression at the high altitude, severing all four of the massive plane's hydraulic stabilizer systems…

Obligatory long form link: https://admiralcloudberg.medium.com/fire-on-the-mountain-the...

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#770

Post by nicbou »

efitz wrote: post #406

The lack of backups makes my blood boil. However, from my own experience, I want to know more before I assign blame. The very first "computer guy" job I had starting in about 1990/1991, my mentor gave me a piece of advice that I remember to this day: "Your job is to make sure the backups are working; everything else is gravy." While I worked in that job, we outgrew the tape backup system we were using, so I started r…

> fortunately my successor found my stack of memos

Those, ironically, were backed up

Post reply on HN