Live data from Hacker News

Citi’s $900M Misfire Happened During Software Switch

bloomberg.com

1–10 of 55 posts

Re: Citi’s $900M Misfire Happened During Software Switch

#3
The title might suggest that the rollout of new software was the issue, but the article states the very contrary: it was the old software that was the culprit:

> An internal review at the bank found humans manually operating the old software were ultimately at fault

Re: Citi’s $900M Misfire Happened During Software Switch

#4
post #3

The title might suggest that the rollout of new software was the issue, but the article states the very contrary: it was the old software that was the culprit: > An internal review at the bank found humans manually operating the old software were ultimately at fault

> An internal review at the bank found humans manually operating the old software were ultimately at fault

Which is of course an entirely bogus cop-out. If a mistake can be made in a manual operation then sooner or later it will be. Lower down the article says that manual checks that were supposed to catch this error failed to do so. Ineffective checks are a management responsibility, and that responsibility goes all the way up to the CEO.

Re: Citi’s $900M Misfire Happened During Software Switch

#6
post #4
post #3

The title might suggest that the rollout of new software was the issue, but the article states the very contrary: it was the old software that was the culprit: > An internal review at the bank found humans manually operating the old software were ultimately at fault

> An internal review at the bank found humans manually operating the old software were ultimately at fault Which is of course an entirely bogus cop-out. If a mistake can be made in a manual operation then sooner or later it will be. Lower down the article says that manual checks that were supposed to catch this error failed to do so. Ineffective checks are a management responsibility, and that responsibility goes all…

At least Citibank is trying to upgrade their ancient systems, but it sure looks as if previous or current CEOs failed to exercise due diligence.

Doesn't look like good risk management at all.

Re: Citi’s $900M Misfire Happened During Software Switch

#7

This is the key part: "But the employee didn’t select the correct system options -- instead allowing the loan to be repaid in full with interest. Colleagues who are supposed to catch such errors didn’t." Saved you a click.

From experience in investigating mishaps like that:

1) no maker-checker control,

2) no imposed limits (with forced maker-checkers - more than one checker)($900m with one click???? what the actual ....),

3) lack of training,

4a) pressure to do this NOW NOW NOW NOW (sorry for the caps),

4b) overworked/tired (matching point 6 below), if that person is "stuck" at home with two screaming kids aged 2-6 for the past five months, I feel for them.

5) toxic environnment that did not allow the employee to spend 2 extra mins to think twice before clicking,

6) in these COVID times not having someone next to him/her and/or was too afraid to ping someone to ask "hey dude, just to make sure, am I using MenuOption1 or MenuOption2 for this almost $1b thingie?" (again, inadequate training & toxic env.)(easier to tap someone in the bag and ask them to look at your screen that get on a Lync call, share screen.

Absolute controls in place would be limits & maker-checker.

And this is the point, when I browse the "jobs" HN, I NEVER see any on audit/controls/GRC.. as if DevOps are the gods of everythinig and auditors are useless and not needed.. sigh

I know there are other (better?) websites when it comes to looking for Audit/Sec work, but I feel that things like that should be taken care of in the development cycle, not the post-mortem of a mishap.

Re: Citi’s $900M Misfire Happened During Software Switch

#8
post #6
post #4

Earlier quoted context omitted.

> An internal review at the bank found humans manually operating the old software were ultimately at fault Which is of course an entirely bogus cop-out. If a mistake can be made in a manual operation then sooner or later it will be. Lower down the article says that manual checks that were supposed to catch this error failed to do so. Ineffective checks are a management responsibility, and that responsibility goes all…

At least Citibank is trying to upgrade their ancient systems, but it sure looks as if previous or current CEOs failed to exercise due diligence. Doesn't look like good risk management at all.

Citi is one of them banks that spend a lot of their $$$$$$ in IT. They jokingly say that they are an IT company with a banking license.

Anyone related can please pitch in with a TA account. How bad/frequent are their Software Errors?

Re: Citi’s $900M Misfire Happened During Software Switch

#9
post #6
post #4

Earlier quoted context omitted.

> An internal review at the bank found humans manually operating the old software were ultimately at fault Which is of course an entirely bogus cop-out. If a mistake can be made in a manual operation then sooner or later it will be. Lower down the article says that manual checks that were supposed to catch this error failed to do so. Ineffective checks are a management responsibility, and that responsibility goes all…

At least Citibank is trying to upgrade their ancient systems, but it sure looks as if previous or current CEOs failed to exercise due diligence. Doesn't look like good risk management at all.

Actually, banks risk management is easy.

Most in-bank or between banks transfers are reversible and usually a non-issue. That why the risk management probably says something like this:

  Risk: Incorrect transfer of funds to customer in another bank
  Mitigation: Manual review of all funds transfer above 5 million dollars
  Mitigation: Besides litigation issues, lost funds are easily recovered by asking the receiving bank
  Status: Risk accepted
Edit: Clarified "Mitigation: Besides litigation issues bank transfers are reversible" into "Mitigation: Besides litigation issues, lost funds are easily recovered by asking the receiving bank"

Re: Citi’s $900M Misfire Happened During Software Switch

#10
post #9
post #6

Earlier quoted context omitted.

At least Citibank is trying to upgrade their ancient systems, but it sure looks as if previous or current CEOs failed to exercise due diligence. Doesn't look like good risk management at all.

Actually, banks risk management is easy. Most in-bank or between banks transfers are reversible and usually a non-issue. That why the risk management probably says something like this: Risk: Incorrect transfer of funds to customer in another bank Mitigation: Manual review of all funds transfer above 5 million dollars Mitigation: Besides litigation issues, lost funds are easily recovered by asking the receiving bank S…

Bank transfers between banks are not reversible.
Post reply on HN