Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire destroys S. Korean government's cloud storage system, no backups available

koreajoongangdaily.joins.com

Post reply on HN
Print view View original

941–950 of 987 posts

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#941

Post by rcxdude »

sleepybrett wrote: post #381
rawgabbit wrote: post #367

Earlier quoted context omitted.

How large is this UPS that a fire can bring down all 96 servers? This story is really unbelievable.

depends on how many batteries were in the facility, if one goes up chances are the rest go too. Can halon systems not put out lithium fires?

Lithium ion batteries provide their own oxidiser, removing oxygen won't put them out (though it will probably help stop the fire from spreading). The only thing that kinda helps is removing the heat (with cold C02 or water, the latter not great for an electrical fire and the former only good for pretty small fires), but that's only a temporary fix usually. Ultimately a lithium battery fire has got to burn itself out.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#942

Post by linksnapzz »

sllabres wrote: post #908
linksnapzz wrote: post #863

Earlier quoted context omitted.

A Geographically-Dispersed Parallel Sysplex for z\OS mainframes, which IBM has been selling since the '90s, can have redundancy out to about 120 miles. At a former employer, we used a datacenter in East Brunswick NJ that had mainframes in sysplex with partners in lower manhattan.

If you have to mirror synchronously the _maximum_ distances for other systems (e.g. storage mirroring with NetApp SnapMirror Synchronous, IBM PPRC, EMC SRDF/S) are all in this range. But an important factor is, that performance will degrade with every microsecond latency added as the active node for the transaction will have to wait for the acknowledgement of the mirror node (~2*RTT). You can mirror synchronously tha…

The actual distance involved in the case of the Brunswick DC is closer to 25 miles to Wall St.; but yes, latency for this is always paramount.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#943

Post by bvan »

How could you even define that as a ‘cloud’? Sounds like good old client-server on a single premise, and no backup whatsoever. Can’t have had very secure systems either.. perhaps they can buy back some of the data off the dark web.. or their next-door neighbor.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#944

Post by garfieldnate »

I know Korea is a fast-changing place, but while I was there I was taught and often observed that the value of "ppalli ppalli" (hurry hurry) was often applied to mean that a job was better done quickly than right, with predictably shoddy results. Obviously I have no insight into what happened here, but I can easily imagine a group of very hurried engineers feeling the pressure to just be done with their G-Drive tasks and move on to other suddenly urgent things. It's easy to put off preparation for something you don't feel will ever come.

I'm going to check all the smoke detectors in my house tomorrow :D

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#945

Post by 0xDEAFBEAD »

pchew wrote: post #790
0xDEAFBEAD wrote: post #640

Earlier quoted context omitted.

Things South Korea is good at producing: Cars, ships, steel, semiconductors, electronics, medicines, tanks, aircraft parts, nuclear reactors... Things South Korea is bad at producing: Software. Not too bad overall.

Interesting interpretation of 'good' in regards to cars.

My parents have been driving the same Hyundai for about 20 years. Never heard them complain about a problem.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#946

Post by mattmaroon »

keepamovin wrote: post #463
mattmaroon wrote: post #401

Earlier quoted context omitted.

It might be different “they”s. Putting on my tinfoil hat, whoever was going to be in hot water over the hack burns it down and now the blame shifts from them to whoever manages G-drive and don’t have a backup plan. Not saying I believe this (or even know enough to have an opinion), but it’s always important to not anthropomorphize a large organization. The government isn’t one person (even in totalitarian societies)…

If there was shady behavior, I doubt it’s about a cyber hack. More likely probably the current administration covering their tracks after their purges. Alternate hypothesis: cloud storage provided doing the hard sell. Hahaha :)

“It’d be a real shame if something happened to your data center…”

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#947

Post by Panzer04 »

nonethewiser wrote: post #887
panda-giddiness wrote: post #744

Earlier quoted context omitted.

Attached end-to-end, they'd extend almost from the Earth to the Sun [1]. Placed in a grid, they'd cover an area larger than Wales [2]. Piled on top of each other, they'd reach a tenth the distance to the moon [3]. --- [1] https://www.wolframalpha.com/input?i=449.5+*10%5E9+*+%28leng... [2] https://www.wolframalpha.com/input?i=449.5+*10%5E9+*+%28area... [3] https://www.wolframalpha.com/input?i=449.5+*10%5E9+*+%28thic..…

I am shocked that 1 and 2 are both true. I would have guessed 1 would have implied a much larger area than Wales.

Funny how unintuitive N^2 growth can be :D

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#948

Post by gtech1 »

KaiserPro wrote: post #728
naruhodo wrote: post #425

Earlier quoted context omitted.

My mind initially went to a government cover-up, but then: > 27th of September 2025, The fire is believed to have been caused while replacing Lithium-ion batteries. The batteries were manufactured by LG, the parent company of LG Uplus (the one that got hacked by the APT). Could the battery firmware have been sabotaged by the hacker to start the fire?

It could have. But replacing a UPS is usually done to right time pressures. the problem is, you can rarely de-energise UPS batteries before replacing them, you just need to be really careful when you do it. Depending on the UPS, Bus bars can be a mother fucker to get on, and of they touch energised they tend to weld together. With lead acid, its pretty bad (think molten metal and lots of acidic, toxic and explosive g…

Don't you have to put UPS's in bypass mode precisely for this reason while doing maintenance on them ?

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#949

Post by pezezin »

rester324 wrote: post #518
pezezin wrote: post #421

Earlier quoted context omitted.

It is the same in Japan. They are really good for hardware and other "physical" engineering disciplines, but they are terrible when it comes to software and general IT stuff. Seriously, I work here as an IT guy and I can't stop wondering how they could become so advance in other areas and stay so backwards in anything software-related except videogames.

Yeah. This is my exact experience too wrt japan! The japanese just somehow can't assess and manage neither the scale, nor the complexity, the risk, the effort or the cost of software projects. Working in japan as a software guy feels like working in a country lagging 30-40 years behind :/

In my experience, risk assessment is the worst part. Japanese culture is extremely risk averse, and the moment you ask them to do something that they have never done before, they freeze in panic. They need a procedure for everything, and have a really hard time improvising.

That's why they are good for industrial processes where they can iterate and improve in small, incremental steps, but terrible for software projects full of uncertainties.

Re: Fire destroys S. Korean government's cloud storage system, no backups available

#950

Post by Intermernet »

famouswaffles wrote: post #817
Intermernet wrote: post #749

Earlier quoted context omitted.

While I understand what you're saying, you can extend this logic to such things as faster-than-light travel, over-unity devices, time travel etc. They're just "hard" math problems. The current state of encryption is based on math problems many levels harder than the ones that existed a few decades ago. Most vulnerabilities have been due to implementation bugs, and not actual math bugs. Probably the highest profile "a…

>While I understand what you're saying, you can extend this logic to such things as faster-than-light travel, over-unity devices, time travel etc. They're just "hard" math problems. No you can't. Those aren't hard math problems. They're Universe breaking assertions. This is not the problem of flight. They're not engineering problems. They're not, "perhaps in the future, we'll figure out..". Unless our understanding o…

According to our understanding of physics, which is based on our understanding of maths, the time taken to brute force a modern encryption standard, even with quantum computers, is longer than the expected life of the universe. The likely-hood of "finding a shortcut" to do this is in the same ball-park as "finding a shortcut" to tap into ZPE or "vacuum energy" or create worm-holes. The maths is understood, and no future theoretical advances can change that. It would involve completely new maths to break these. We passed the "if only computers were a few orders of magnitude faster it's feasible" a decade or more ago.
Post reply on HN