Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Banks scramble to fix old systems as IT 'cowboys' ride into sunset

reuters.com

Post reply on HN
Print view View original

101–110 of 361 posts

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#101

Post by Johnny_Brahms »

acchow wrote: post #93
sanguy wrote: post #52

Been working on accounting systems in RPG and COBOL since ~1992. I also know C/X86ASM/Pascal/Delphi/VB/Fortran. Never bothered with C++ that much; played with Java a bit but Oracle irritates my bowels so moved away from that. As mentioned in the article it's good work; but it is also not easy work. You tend to go through cycles of being pushed out to brought back under extreme emergency at any costs to get stuff work…

> high 6 figures a year without working too hard Wow, like $800-900k?

It's not unheard of. But you must understand that these are people that have a pretty decent job security in an often huge codebase that has been aquiring cruft for 30 years in language most people don't understand. I would like to be able to claim that the code is good, but some of the people writing it in the first place were not programmers. It is not strictly bad, but some parts are just mine fields.

A bit related:

I spent 2 years of my life on a C codebase written by mathematicians that would much rather have written it in APL. I don't know if you have seen C written as APL, but that is 2 years of my life I will never get back. I left for a lesser paid, more fun job. Upon leaving the manager offered me a 60% pay raise (yup, I should have had higher demands, but at least I proved myself :) ).

Unless you enjoy torturing yourself, legacy COBOL programming is not very rewaeding, and at least in Sweden, most of it is slowly being moved to other languages.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#102

Post by UweSchmidt »

movedx wrote: post #54
sanguy wrote: post #52

Been working on accounting systems in RPG and COBOL since ~1992. I also know C/X86ASM/Pascal/Delphi/VB/Fortran. Never bothered with C++ that much; played with Java a bit but Oracle irritates my bowels so moved away from that. As mentioned in the article it's good work; but it is also not easy work. You tend to go through cycles of being pushed out to brought back under extreme emergency at any costs to get stuff work…

I smiled at your last comment. You're clearly a charming individual :-) Out of interest, what's the solution here? With your experiences in mind, what are the banks doing wrong from a technological perspective? I get the impression banks should almost start again on the side, building a completely new bank they then (manually?) move customers over to, or just use to take in new customers whilst waiting for the previo…

You can't wait for customers to "die out" as their mortgages will go on for decades. You definitely need a customer's old data in the new system. Now you have to maintain and develop two systems - bugs, banking regulation require constant work. Often the clock is ticking for old systems and asking vendors to provide and maintain ancient tools can be expensive or impossible. You can't "manually" move customers over, you need to "migrate" lots and lots of data - both systems will have huge, completely different database schemas.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#103

Post by grogenaut »

flukus wrote: post #55
dsr_ wrote: post #10

The problem is technical debt. Banks didn't realize they were software houses, they thought that IT was a tool.

It is amazing how many companies don't think they're software companies, even though they are 100% reliant on their in house software. "We aren't a software company" is even a direct excuse I've heard a few times to justify ignoring technical debt.

What's worse is companies who 100% outsource to customized versions of a vendor's code. At least banks own the source to what they run even if they can't run it without a license. Clothing company I know sitting next to amazon pays the equivalent of about 5 fte senior amazon engineers a year in licensing the software they run every year to a vendor who charges them another 5 engineers worth to implement all of the customizations they want and "re-negotiates" the contract every 3 years. I say negotiate because there's not really an alternative to switch, it'd be a 3 year conversion project and they refuse to hire devs.

They gave a 8k raise to most tech folks about 5 years ago because they were bleeding people across the street. Which brought their senior tech folk to within 20k of an amazon SDE intern salary. People still left.

I'm biased as an engineer but I'm not really sure why they keep this relationship going.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#104

Post by UweSchmidt »

Johnny_Brahms wrote: post #101
acchow wrote: post #93

Earlier quoted context omitted.

> high 6 figures a year without working too hard Wow, like $800-900k?

It's not unheard of. But you must understand that these are people that have a pretty decent job security in an often huge codebase that has been aquiring cruft for 30 years in language most people don't understand. I would like to be able to claim that the code is good, but some of the people writing it in the first place were not programmers. It is not strictly bad, but some parts are just mine fields. A bit relate…

Best story I've heard is one of the old programmers commenting his COBOL code in latin. It was a different time...

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#105

Post by tyingq »

kpil wrote: post #99
jerguismi wrote: post #95

Earlier quoted context omitted.

What about not doing a rewrite? What about instead refactoring, documenting etc the old system.

Because COBOL is the PHP of the 60s, and mainframes are slow and expensive. Also, too many of the talents are stuck in a blocked I/O mindset somehow. Some are wizards though, writing assembly and making raspberry pi sized systems blazingly fast. OK a couple of raspberry pi:s

As a counterpoint, many very well funded and talent rich organizations have failed to retire what TPF mainframes do every day for airlines, banks, and credit card companies.

Cobol isn't involved, but those slow mainframes are.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#106

Post by grogenaut »

doggydogs94 wrote: post #97

Just went into Autozone and the interface looked like a IBM CICS screen scraper. Went to my insurance agent; we looked at nice screens, but when real money became involved, we popped into this ugly legacy screen.

Or maybe it's because keyboard input is faster for a lot of heavily repeated form type things than a web interface.

Web is nice when users don't know how to use it. When they do they're not really looking 1/2 the time.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#107

Post by gambler »

acchow wrote: post #93
sanguy wrote: post #52

Been working on accounting systems in RPG and COBOL since ~1992. I also know C/X86ASM/Pascal/Delphi/VB/Fortran. Never bothered with C++ that much; played with Java a bit but Oracle irritates my bowels so moved away from that. As mentioned in the article it's good work; but it is also not easy work. You tend to go through cycles of being pushed out to brought back under extreme emergency at any costs to get stuff work…

> high 6 figures a year without working too hard Wow, like $800-900k?

This seemed surprising to me on the first read, but it makes sense. An average senior software engineer probably makes ~$120K in US. They often work on systems that aren't in any way critical and are more or less replaceable. Anything in COBOL these days is by definition is crucial and highly expensive infrastructure. The talent pool to work with that tech is small. They are by definition very experienced and probably highly specialized even within that pool. So upper six digit salary actually makes sense.

To be honest, the idea of companies paying through their noses because of decades of short-term thinking makes me smile. Karma in action.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#108

Post by ternaryoperator »

acchow wrote: post #93
sanguy wrote: post #52

Been working on accounting systems in RPG and COBOL since ~1992. I also know C/X86ASM/Pascal/Delphi/VB/Fortran. Never bothered with C++ that much; played with Java a bit but Oracle irritates my bowels so moved away from that. As mentioned in the article it's good work; but it is also not easy work. You tend to go through cycles of being pushed out to brought back under extreme emergency at any costs to get stuff work…

> high 6 figures a year without working too hard Wow, like $800-900k?

I expect he meant the high range of 100K-200K. It's a colloquialism I hear periodically. I took it to mean $180K-$200K.

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#109

Post by ltratt »

My gut feeling is that one can minimally maintain systems in "forgotten" languages indefinitely if they're basically stable and feature complete. But it seems that people come unstuck for systems that still need to evolve -- even if that evolution is slow and/or sporadic. The obvious alternative is to rewrite systems in new languages, but that is generally prohibitively expensive. Even for those that can afford the expense, the new system tends to not to take considerable time to bed in, so flipping the switch from the old to the new is only for the brave or those with excellent PR departments.

Although we didn't have this in mind when we started on this work, I've come to think that language composition might offer a way out of this mess (I immodestly offer [1] as an early example). If one composes an old and a new language, one can migrate a system from an old to a new language bit-by-bit. As well as amortising the translation cost, it also means that one can minimise the "switch flipping" problem: disasters seem much less likely if the composed system can be introduced gradually.

[1] http://tratt.net/laurie/blog/entries/fine_grained_language_c...

Re: Banks scramble to fix old systems as IT 'cowboys' ride into sunset

#110

Post by gambler »

sanguy wrote: post #52

Been working on accounting systems in RPG and COBOL since ~1992. I also know C/X86ASM/Pascal/Delphi/VB/Fortran. Never bothered with C++ that much; played with Java a bit but Oracle irritates my bowels so moved away from that. As mentioned in the article it's good work; but it is also not easy work. You tend to go through cycles of being pushed out to brought back under extreme emergency at any costs to get stuff work…

> Companies never think of the old guys as the ones to implement the new system - that's a job for the "enterprise experts"

So true, even for more recent stuff. It's so absurd I always wanted to make a web comic about this. Companies keep ignoring advice of their own developers, and then eventually hire some "technology expert" who is going to implement the same tech their existing staff recommend years ago. Except, of course, the expert has no idea about business processes and user needs, so you end up with a long and expensive train-wreck that results in something barely better than what you had before.

Post reply on HN