Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

nextgov.com

Post reply on HN
Print view View original

171–180 of 211 posts

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#171

Post by beezle »

DonaldFisk wrote: post #155
yazaddaruvala wrote: post #149

Earlier quoted context omitted.

This is a relatively solved problem. Abstractions. 50 years ago the IRS had to build a webserver, the servlet application code, it had to build in fault tolerance without SQS or Kafka. If built today, the IRS' entire system would probably run on Rails with JRuby, deployed with Docker and K8s, and be 1000x more maintainable.

50 years ago the WWW didn't exist and there was no such thing as a servlet. The system ran on a mainframe not 50 but 60 years ago. The only high level language at the time was FORTRAN, but this system was apparently written in assembly language. Does anyone know the type of computer it ran on? The article it linked to says IBM mainframe but which model?

COBOL, ALGOL, PL/1

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#172

Post by Tempest1981 »

mbell wrote: post #32
johnklos wrote: post #18

Something that the kids of IT today will never learn: Never replace anything unless the thing doing the replacing is better than the thing being replaced. The IRS' IT "upgrade" is just a huge money grab. Nobody has the slightest interest or desire to do anything properly - they simply want in on that lucrative IRS money. It's like the saying: "Anyone who is capable of getting themselves made President should on no ac…

Or you can continuously keep your code / systems up to date and never get wedged into a nearly impossible situation of having to rewrite 60 year old code (which likely has nothing resembling a test suite nor even a spare piece of compatible hardware to test on).

Is the 60-year old assembly code updated as the tax laws change? I guess it must be.

Before embarking on a re-write, how about we create a simplified tax code (laws)? Not sure which task is tougher.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#173

Post by fouc »

ChuckMcM wrote: post #93
raverbashing wrote: post #34

The question for the industry remains: how do we deal with systems that should be up for 10 years? 25 yrs? 50 yrs? 100 yrs? We didn't even have computers as they exist today 50 years ago. How do we separate business logic in a self-contained way that can be independent of hardware and (from a certain level) software?

We have shown pretty clearly that any computer we build at day 1 can be emulated perfectly by a computer built day 1 + 50 years. However, building systems that can evolve is a larger issue. The nut here is that these systems evolve in a very haphazard way do to funding availability, scaling stresses, and changing requirements. For the IRS, I think it would be useful to engage the US Digital Service to see if they cou…

It occurred to me just now that the US Digital Service will become increasingly powerful as it helps digitizes the government, and 30 years from now it could be synonymous with the government.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#174

Post by hermitdev »

gascan wrote: post #26
favorited wrote: post #2

Fun (scary) fact – significant portions of the IRS tax system is still written in assembly. It's basically the original codebase plus 40 years of additions and patches as the tax code changed. They were working on projects to "elevate" it to Java, because it's pretty difficult to maintain it as-is.

I mean, maintainability aside they are a giant calculator right? You don't need to be running the latest, hippest web platform to do a good job at doing a lot of math. And, will any of the current en-vogue tech still be working in 50 years? If you had to pick, Java does make a lot of sense. Java & C are safe bets for "languages that will stick around for decades".

Having worked on financial code that is brittle and subject to far less minutiae than the multitude of IRS code, I can be very sympathetic on the complexity involved. It is hard to handle all of the complexity in a generic maintainable manner. I once ran into a 56-page printed function to calculate the margin required for a portfolio at a prime broker. Was it poorly written? Yes. Was it also representative of the complexity of the problem? Yes. Not an excuse for poor design, though.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#175

Post by OrganicMSG »

raverbashing wrote: post #34

The question for the industry remains: how do we deal with systems that should be up for 10 years? 25 yrs? 50 yrs? 100 yrs? We didn't even have computers as they exist today 50 years ago. How do we separate business logic in a self-contained way that can be independent of hardware and (from a certain level) software?

Code all long term business logic in magnetic core memory running on hardened valves with dixie tube displays.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#176

Post by colechristensen »

raverbashing wrote: post #34

The question for the industry remains: how do we deal with systems that should be up for 10 years? 25 yrs? 50 yrs? 100 yrs? We didn't even have computers as they exist today 50 years ago. How do we separate business logic in a self-contained way that can be independent of hardware and (from a certain level) software?

>How do we separate business logic in a self-contained way that can be independent of hardware and (from a certain level) software?

Come up with programming languages with limited domain specific functionality that don't allow you to do funny machine specific things. (or even ordinary things like pointers)

Use the restricted languages for business logic and have a second language to do the plumbing.

Or just target a generic x86_64 virtual machine.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#177

Post by mythrwy »

_RPM wrote: post #39
favorited wrote: post #2

Fun (scary) fact – significant portions of the IRS tax system is still written in assembly. It's basically the original codebase plus 40 years of additions and patches as the tax code changed. They were working on projects to "elevate" it to Java, because it's pretty difficult to maintain it as-is.

Should it be written in Go, or Rust?

Haskell. Obviously.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#178

Post by empath75 »

sehugg wrote: post #164

A delivery date for CADE 2, the IRS’ subsequent modernization effort, has slipped several years even as contractors working on the project have earned as much as $290 million. They call them "Beltway Bandits" for a reason.

As someone working on a modernization contract, the problems are more complicated than just contractors standing on shovels.

Building an app to a complex and ever shifting spec is a much different process than the way that startups work. You can’t just release a half done app to friends and family. You need to do the whole thing at once, with 100% if the functionality and it has to be ready for everyone in the country to use it in day 1. And the federal government has fairly onerous requirements for getting stuff through security— you can’t just google for random shit from GitHub and expect to put it straight into production.

I can only speak to my particular project, but everyone on it is working hard at getting it done on time and on budget, using modern deployment and development practices, but there’s limits to how much you’re allowed to just move fast and break things.

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#179

Post by empath75 »

yazaddaruvala wrote: post #149
raverbashing wrote: post #34

The question for the industry remains: how do we deal with systems that should be up for 10 years? 25 yrs? 50 yrs? 100 yrs? We didn't even have computers as they exist today 50 years ago. How do we separate business logic in a self-contained way that can be independent of hardware and (from a certain level) software?

This is a relatively solved problem. Abstractions. 50 years ago the IRS had to build a webserver, the servlet application code, it had to build in fault tolerance without SQS or Kafka. If built today, the IRS' entire system would probably run on Rails with JRuby, deployed with Docker and K8s, and be 1000x more maintainable.

From my experience the government is excited about kubernetes and openshift and are slowly starting to use it for prod (security approval is tricky)

Re: IRS’ 60-Year-Old IT System Failed on Tax Day Due to New Hardware

#180

Post by DonaldFisk »

beezle wrote: post #171
DonaldFisk wrote: post #155

Earlier quoted context omitted.

50 years ago the WWW didn't exist and there was no such thing as a servlet. The system ran on a mainframe not 50 but 60 years ago. The only high level language at the time was FORTRAN, but this system was apparently written in assembly language. Does anyone know the type of computer it ran on? The article it linked to says IBM mainframe but which model?

COBOL, ALGOL, PL/1

Not in 1958.

The first COBOL compiler was available 2 years later in 1960. Algol 58 wasn't available until 1960 either.

The ALGO (Algol 58) manual for the Bendix: http://www.piercefuller.com/collect/bendix/algo6008.pdf

The first compiler for PL/1 was delivered much later, in 1966.

You could have suggested LISP 1.0 but that appeared in 1960 too (as an interpreter, the first compiler was written in 1962).

LISP, COBOL, ALGOL: 1960 was a remarkable year in the history of programming languages.

Post reply on HN