Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Was the Y2K crisis real?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 385 posts

Re: Ask HN: Was the Y2K crisis real?

#191

Post by __d »

I worked at a regional bank. Like many banks, we offered mortgages, so starting in 1970, the 30 year mortgages had a maturity date in 2000, and our bank had begun the process of adapting its systems from 2-digit to 4-digit dates.

Basically all of our software was written in COBOL, and most COBOL data is processed using what we'd consider today to be string-like formats. And to save space (a valuable commodity when DASD (aka hard drives) cost hundreds of thousands of dollars, and stored a few megabytes of data) two-digit dates were everywhere.

I started in 1991. The analysis had been done years before, and we knew where most of the 2-digit problems were, so it was just a matter of slowly and steadily evolving the system to use 4-digit dates where possible, or to shift the epoch forward where that made sense.

Every few months we'd deploy a new version of some sub-system which had changed, migrate all the data over a weekend, and cross off another box in the huge poster showing all the tasks to be done.

External interfaces were the worst. Interbank transfers, ATM network connections, ATM hardware itself, etc, etc. We mostly tried to switch internal stuff first but leave the APIs as 2-digit until the external party was ready to cut over. Similarly between our internal systems: get both ready internally, migrate all the data, and then finally flick the switch on both systems to switch the interfaces to 4-digit.

Practically, it meant that we our development group (maybe 30 people?) was effectively half that size for 5 or 6 years in the early 90's as the other half of the group did nothing but Y2K preparation.

All of these upgrades had to be timed around external partners, quarterly reporting (which took up a whole weekend, and sometimes meant we couldn't open the branches until late on the Monday after end-of-quarter), operating system updates, etc, etc. The operations team had a pretty solid schedule booked out years in advance.

We actually had two mainframes, in two data centers: one IBM 3090 and the other the equivalent Armdahl model. We'd use the hot spare on a weekend to test things.

It was a very different world back then: no Internet, for a start. Professional communication was done by magazines and usergroup meetings. Everything moved a lot slower.

I left that job before Y2K but according to the people I knew there, it went pretty well.

Re: Ask HN: Was the Y2K crisis real?

#192

Post by dejv »

was8309 wrote: post #104
_red wrote: post #64

Earlier quoted context omitted.

>dbase At the time I was managing a dBase / FoxPro medical software package...we were a small staff who had to come up with Y2K mitigation on our own. Our problem is we only had source code for "our" part of the chain...other data was being fed into the system from external systems where we had no vendor support. Thus our only conceivable plan was to do the old: If $year It worked in 99.9% of the cases which was enou…

and it was pretty embarrassing when I forgot to update this same thing in 2010

Man, I've just remember that I had to fix this kind of bug in 2010 and I am sure I just bumped the number to 20. I guess somebody had to be fixing it lately, I just hope they didn't just replace 2 with 3.

Re: Ask HN: Was the Y2K crisis real?

#193

Post by unoti »

I worked hard on y2k issues in 1998-1999. It was a real thing for my company at the time. It was a crisis averted. In the 80’s and 90’s I worked on many systems where the equivalent of “max date” or “end of time” was expressed as 12/31/99. The way that these systems expressed, stored, entered, and validated dates all had to be reworked in a series of major overhauls.

Re: Ask HN: Was the Y2K crisis real?

#194

Post by GekkePrutser »

It's like the Ozone layer and acid rain.

Climate sceptics often use these as an excuse. "Yes but there was sooo much hubbub about those and it proved to be nothing".

Well, yes it is nothing now because it was decisively and effectively handled. The ozone layer is still recovering but on a steady path there, and acid rain is reduced to the point of not really being an issue anymore (at least in the western world!).

Stuff really would have gone wrong with Y2K. Maybe not armageddon but yes. There was a problem.

Re: Ask HN: Was the Y2K crisis real?

#195

Post by chungus_khan »

rriepe wrote: post #9
dhosek wrote: post #5

I was working at a telecommunications startup in 1999. They were founded in 1997. A big part of what I was doing was fixing Y2K bugs. That said, none of the bugs would have been critical to the operations of the services. Everything was in the billing systems and I think if unfixed it would have been more of a reputation hit than anything. Also, "begs the question" doesn't mean what you think it means. https://en.wik…

Personally, I've given up on begging the question. It just means "raises" now. The descriptivists always win in the end.

The descriptivists don't even win, they just accept reality. Language is not constructed by individual humans or small groups, but socially over entire societies. Fighting this makes prescriptivists look like grumpy armchair academics and still doesn't stop me from saying "le weekend".

Re: Ask HN: Was the Y2K crisis real?

#196

Post by Diederich »

pjmorris wrote: post #170
wglb wrote: post #163

Earlier quoted context omitted.

While I don't have direct knowledge of Bentonville, the general impression that I have formed is that they are one or two cuts above any other enterprise of their size.

I once worked with an F500 retailer who had Bentonville 'graduates' among their IT staff. Sharp people among other sharp people. It takes a lot to keep the lights on pretty much everywhere. Walk in to any big box store and look around for how many gadgets and systems there are.

Most everyone would be amazed at much networked equipment is in a WalMart supercenter. By the time I left, there were four routers, four big core switches, a couple dozen 'edge' switches, and scores of access points. And that's just the networking gear. As of 2009, my team's network discovery system had identified tens of millions of separate IP addressable nodes on the global network.

At that scale, everything had to be almost perfectly automated.

And, to be clear, this was not a static, boring network. Stores, home offices, distribution centers, you name it, were being added, moved, expanded, deallocated 24 hours a day, all over the world.

And the device configs were far from cookie cutter. An eye-wateringly complex heuristic was required to dynamically create router, switch and access point config that would allow a given store to function.

And, one quarter in the early 2000s, we (Network Engineering) achieved six sigma (99.9999%) global uptime.

Things did start going downhill around 2004 or so, a few years after Kevin Turner became CIO.

Re: Ask HN: Was the Y2K crisis real?

#197

Post by Diederich »

cjohnson318 wrote: post #189
Diederich wrote: post #153

Earlier quoted context omitted.

I've worked in silicon valley for the past 11 years, in startups as well as two so-called FAANG companies. There are a lot of brilliant people out here. So that is well known. What's not well known is this: there are pockets of brilliant people elsewhere, including Bentonville, Arkansas. Especially in the years running up to 2000, I had the pleasure and honor of working with some of the most talented people in the wo…

I knew a guy that worked at Walmart for years, right out of college. He said that the biggest issue he had with onboarding people was impressing upon them the scope of Walmart's enterprise. You don't just "move fast and break stuff". If your clever hack causes an outage in pharmacy, then you're directly affecting millions of people across the country that need medicine.

Yup.

It's funny, because for the first 2/3 of my tenure there, WalMart Information Systems Division did move extremely fast....and yet almost never broke anything.

It was an amazing thing to see.

Re: Ask HN: Was the Y2K crisis real?

#199

Post by Diederich »

pjmorris wrote: post #170
wglb wrote: post #163

Earlier quoted context omitted.

While I don't have direct knowledge of Bentonville, the general impression that I have formed is that they are one or two cuts above any other enterprise of their size.

I once worked with an F500 retailer who had Bentonville 'graduates' among their IT staff. Sharp people among other sharp people. It takes a lot to keep the lights on pretty much everywhere. Walk in to any big box store and look around for how many gadgets and systems there are.

> I once worked with an F500 retailer who had Bentonville 'graduates' among their IT staff.

May I ask which one? Just idle curiosity.

Re: Ask HN: Was the Y2K crisis real?

#200

Post by ninju »

Yes..the Y2K was very real and was (moderately) well handled.

The potential for large disruptions in financial, real-time and other systems would have occurred if not for the effort applied.

Unfortunately some problems require a certain level of media-awareness and/or hysteria before we devote the necessary resources to fix the problem before it become a crisis

Post reply on HN