Live data from Hacker News

Tell HN: Wells Fargo completely offline

news.ycombinator.com

51–60 of 256 posts

Re: Tell HN: Wells Fargo completely offline

#51
post #21

You can still access Wells Fargo Online via the direct link: https://connect.secure.wellsfargo.com/auth/login/present?ori...

Confirmed - was able to log in. Some account info is still there, though my mortgage is unavailable. Maybe they'll just lose it?

Your deposits, sure, your trust, of course. The mortgage loan? Fat chance.

Re: Tell HN: Wells Fargo completely offline

#53
post #12

Apparently there was a fire in a Datacenter - theres a thread on /r/sysadmin by an insider. https://np.reddit.com/r/sysadmin/comments/ao4g2y/wells_fargo...

Incredible. They had all their mission-critical infrastructure in a single data center. How many billions of dollars do they make per year? And they can't afford even a tiny bit of redundancy? If I were a customer, I'd use this as a sign that this company is not technically competent enough to manage my money.

Every major financial regulator has business continuity and disaster recovery requirements but the standards are woefully outdated. A plan that gets the bank back to full functionality within 24 hours would be acceptable to most regulators and even considered speedy to some.

Tangentially related, I highly recommend the movie "Out of the Clear Blue Sky." Cantor Fitzgerald was a bond trading firm at the top of one of the twin towers and lost every employee who was in the office on 9/11. Incredibly, despite losing the majority of their employees and despite losing almost all of their trading infrastructure, they managed to resume operations in time for the bond market's reopening 48 hours later.

Re: Tell HN: Wells Fargo completely offline

#54

Earlier quoted context omitted.

are there non liquid fire suppression systems such as pulling all the oxygen out of a room Yes. One place I worked (not a tech company, but with tons of electronics), when the fire alarms went off we had xx seconds (I don't remember the number) to get out of the building before something called Intergen was vented into the room to somehow suck all of the oxygen out, and if we were still inside we'd be dead. It must b…

This page suggests that Inergen (unless there is a thing called Intergen that I'm confusing it with?) mixes with air to lower the oxygen content, but still remain breathable: http://www.tyco.no/products/Gaseous-Fire-Suppression/inergen...

It's definitely not good to breathe in though, so you can still see why, for liability reasons, the company would want all employees to evacuate if it's going to be deployed, and leave it to firefighters with SCBA equipment.

Re: Tell HN: Wells Fargo completely offline

#55
post #12

Earlier quoted context omitted.

Incredible. They had all their mission-critical infrastructure in a single data center. How many billions of dollars do they make per year? And they can't afford even a tiny bit of redundancy? If I were a customer, I'd use this as a sign that this company is not technically competent enough to manage my money.

If you're waiting for this to be the straw to move away from Wells Fargo after the last set of scandals, you may have some issues...

Unfortunately they're the holders of my mortgage (it was sold twice and they ended up with it). So I don't even have a choice of being their customer for the next ~25 years.

Re: Tell HN: Wells Fargo completely offline

#56
post #12

Apparently there was a fire in a Datacenter - theres a thread on /r/sysadmin by an insider. https://np.reddit.com/r/sysadmin/comments/ao4g2y/wells_fargo...

Incredible. They had all their mission-critical infrastructure in a single data center. How many billions of dollars do they make per year? And they can't afford even a tiny bit of redundancy? If I were a customer, I'd use this as a sign that this company is not technically competent enough to manage my money.

Incredible. You made up a really stupid and (for a bank) illegal thing, then said it's stupid!

If I were a company, I'd use this as a sign you are not competent enough to receive our money.

No dipshit. They have active->sync->async or star replication, live every other bank. Guess what - when you actually fails over it's not always perfectly smooth. The OP made up that "all of wells fargo" went offline, when in reality tiny bits and pieces die. Have you ever worked or did you wander off the high school tech subreddit? When you actually fail over, sometimes not everything fails over smoothly and you have to do database media recovery and replay redo logs.

Re: Tell HN: Wells Fargo completely offline

#57
Regional bank checking in. We've been told to reroute certain workloads through other means until 6am central Friday. Regardless of that, the story coming out of this over the next few days is really going to help Business Continuity departments at other institutions plan for this scenario. My teams are running mock scenarios this afternoon based on the auto power shutdown causing a data center to be unable to fail over. Might end up being something else entirely, but it's still a good scenario to investigate.

Re: Tell HN: Wells Fargo completely offline

#58

Earlier quoted context omitted.

are there non liquid fire suppression systems such as pulling all the oxygen out of a room Yes. One place I worked (not a tech company, but with tons of electronics), when the fire alarms went off we had xx seconds (I don't remember the number) to get out of the building before something called Intergen was vented into the room to somehow suck all of the oxygen out, and if we were still inside we'd be dead. It must b…

This page suggests that Inergen (unless there is a thing called Intergen that I'm confusing it with?) mixes with air to lower the oxygen content, but still remain breathable: http://www.tyco.no/products/Gaseous-Fire-Suppression/inergen...

The resultant 12% oxygen is 60% of normal atmospheric air. It's still breathable, if you're an average healthy person with healthy cardiac and pulmonary function, sitting at rest - but it's not healthy, it's enough that you're going to start seeing systemic responses. The reduction in partial pressure of oxygen being sucked into the lungs will also cause an increase in the partial pressure of carbon dioxide and inert gases.

I think the concern likely comes from:

-Folks that are of poor cardiac function are going to be evacuating, meaning increased cardiac demand under stress, while being somewhat oxygen-starved. This could tip some folks into an acute episode that otherwise wouldn't.

-Folks that are of poor oxygen function, who are borderline hypoxemic to begin with. Think folks with chronic obstructive pulmonary disorder: about 5% of your employees aged 55-65 will have it.

You won't suddenly kill a building full of people. I'm guessing the evacuation rush is to make sure they're not liable for unnecessarily sending a couple to the hospital.

Re: Tell HN: Wells Fargo completely offline

#59

Earlier quoted context omitted.

If you're waiting for this to be the straw to move away from Wells Fargo after the last set of scandals, you may have some issues...

I just started a new job, so I'm just going to set up an account at my local CU and have my direct deposits go there. Guess I'll have to close my WF accounts once they get this mess sorted out!

This is one of the best financial decisions you will ever make.

Re: Tell HN: Wells Fargo completely offline

#60
post #28
post #19

4chan's pol forum is all over it. Apparently it's a "happening" worthy of hundreds of comments

Really? /pol/? this is what the merry band of racist edgelords have fixated on? Seems like it would be more /g/'s thing.

Anything that they can spin into some insane fascist conspiracy theory.
Post reply on HN