Live data from Hacker News

Ask HN: Was the Y2K crisis real?

news.ycombinator.com

351–360 of 385 posts

Re: Ask HN: Was the Y2K crisis real?

#351

Earlier quoted context omitted.

Worked in a project to migrate a huge and critical .gov database from an AS400 nobody understood anymore to Microsoft SQL Server. We had to guess some of the values in some bitwise fields through trial and error. Made it with a few weeks to spare - nobody noticed.

I'm quite surprised no-one understood an AS/400 in 2000 - they were still pretty current and an important part of the IBM range, surely.

Understanding the OS of the AS/400 is a far cry from understanding all the custom software running on it.

Re: Ask HN: Was the Y2K crisis real?

#352

Earlier quoted context omitted.

Worked in a project to migrate a huge and critical .gov database from an AS400 nobody understood anymore to Microsoft SQL Server. We had to guess some of the values in some bitwise fields through trial and error. Made it with a few weeks to spare - nobody noticed.

I'm quite surprised no-one understood an AS/400 in 2000 - they were still pretty current and an important part of the IBM range, surely.

Oh, understanding the underlying system, that's comparatively easy. Understanding the bespoke business logic and how it's represented in the system...now that's harder, an order of magnitude at least.

Story time: years ago (2010-ish?), we were doing some light web CMS work for a client. Nothing too complex, except we had to regenerate one section off a data feed that we received in a very peculiar custom format.

Great, everything worked, we finished on time and on budget. And then: "oh, and could you also check this other site? We need some minor tweaks on what your predecessors made". That other site was also consuming that data feed, so we took a peek. It ran in PHP3, walled off from everything, processed its own intermediate languageS, and the output was a massive 200+ page PDF (which was then manually shipped to offset print in large quantities). For Reasons, this had to run daily and had to work on first try. There was no documentation, no comments, and no development environment: apparently this was made directly on prod and carefully left untouched.

Needless to say, the code was massively interdependent, so that the minor tweaks were actually somewhat harder. We did manage to set up a development VM though, and document it a bit - but last I checked, the Monstrous Book Of Monsters Generator still seems to be chugging along in its containment.

Re: Ask HN: Was the Y2K crisis real?

#353

Earlier quoted context omitted.

That's a fantastic story, I'm impressed the professor didn't try to take the whole settlement.

It's sad to live in a world where people are impressed when someone does the right thing.

It’s sad to live in a world where people are naive enough to assume that doing the right thing without incentive is commonplace. You must be great to negotiate with.

Re: Ask HN: Was the Y2K crisis real?

#354

Earlier quoted context omitted.

Most everyone would be amazed at much networked equipment is in a WalMart supercenter. By the time I left, there were four routers, four big core switches, a couple dozen 'edge' switches, and scores of access points. And that's just the networking gear. As of 2009, my team's network discovery system had identified tens of millions of separate IP addressable nodes on the global network. At that scale, everything had t…

Yeah walmart is a really beautifully ugly beast of a company. On all levels, not hust tech... Ill have to look into kevin turner though, i dont know who that is.... Anything you’d care to share/point at?

https://en.wikipedia.org/wiki/B._Kevin_Turner is the guy but it doesn't give any mention as to why I (and many of my peers) believe that his tenure as Walmart CIO had a strong negative influence on ISDs trajectory.

I don't have much time right now, but I'll share a moment when I saw him interacting with some people during a crisis.

We got hit really hard with https://en.wikipedia.org/wiki/Code_Red_(computer_worm) even though only a minority of our systems were running Windows at the time. There were, at the time, a pair of Windows servers in each store, and each store was connected via a 56k frame relay circuit to the home office, along with a modem connected to a standard phone line for dial backup in case that was down.

Anyway, Code Red rapidly spread through all of these servers, and together they created enough traffic to basically bring down the frame relay network. We were, at the time, slightly over-subscribed at the T1 termination points, so if most of the stores on a termination leg maxed out their individual 56k links at the same time, the whole leg would be effectively down.

(Apologies for the unexpected dive into old networking stuff...it kind of just came out. :)

So most of our stores were down hard, meaning that most card transactions would not work. There was a facility to allow credit cards to go through without verification with a relatively low charge ceiling. But especially back then most transactions were NOT credit, and so weren't working.

This was a bad, bad situation, and it persisted for a number of days. It might have garnered some national news attention.

There were a number of war rooms going on. My team was in one, and we were working towards a solution by using a transient dial backup connection to reach into the store router and apply a draconian ACL.

ANYway, back to Kevin Turner. We saw him talking to a group of VPs and directors outside of our war room. He dismissed most of them, except for one, with whom he continued to have a very heated conversation.

Turner was definitely the aggressor; he got closer and closer to this person, got louder and louder. He ended up backing him against a wall and shouting in his face for a few minutes before walking away, jabbing his finger in his chest.

Just one story. We didn't like KT very much.

Re: Ask HN: Was the Y2K crisis real?

#355

Earlier quoted context omitted.

> people literally spent tens of billions of dollars in effort to fix it The sceptic inside me is curious as to how much they actually accomplished in their effort to fix it. I mean, yes, they spent billions to get millions of lines of code read, but how many fixes have been made and what would be the cost (when compared to those billions of dollars spent) if they weren't fixed at all.

Lots of fixes were made. Everything would have broken if they hadn't been fixed. Why are you sceptical about this?

Lots of people seem to come up with it now. It's been two decades, and it's brought up by people who use it as an example in their other other agendas:

"look at y2k: we spent so much money, and nothing happened!" (so far, so good) "...therefore, if we hadn't, nothing would have happened anyway!!" (uh, based on...?) "...and it was all a massive scam, just like any massive present issue!!!" (ah, there's the underlying reason)

Re: Ask HN: Was the Y2K crisis real?

#356
post #329
post #303

Meanwhile, Linux didn't move the time_t to 64bit despite. They're still working on it now, and not fully prepared. The BSDs fare much better on that, as most of them have done this a long time ago.

That's the curse of backwards compatibility. AFAIK, on BSDs not only are the kernel and the C library developed together (so neither needs to stay compatible with older versions of the other), but also programs are expected to be recompiled for every major release of the operating system (so there's no need to stay compatible with binary-only software from the 90s).

Yet Linux broke things several times around those times. I recall Linux 2.0 was very breaking. gcc 3.1 -> 3.2 also broke almost everything a few years later.

Linux has had plenty of opportunity to fix stuff, all wasted.

Re: Ask HN: Was the Y2K crisis real?

#357
My impression is the same as dwheeler. Y2K could have been a crisis were it not for the swift, expensive and sometimes heroic actions of coders who saved us from a calamity. I don't know that putting gallons of water in the basement was a necessity - water treatment plants probably would have still functioned - but even those plants are controlled by computers, so I'd say it was collective action.

Re: Ask HN: Was the Y2K crisis real?

#358

Earlier quoted context omitted.

From what I've heard, a number of IT departments used it as justification to dump legacy systems. For example, at a former employer of mine, a big justification for getting rid of the IBM-compatible mainframe and a lot of legacy systems which ran on it (various in-house apps written in COBOL and FORTRAN) was Y2K. In reality, they probably could have updated the mainframe systems to be Y2K-compliant. But, they didn't…

COBOL programmers were charging a fortune to be hauled out of retirement to work on this. There was a huge shortage of experienced COBOL devs. And devs who actually understood the specific legacy system involved were even rarer. If the company had customised their system and not kept the documentation up to date or trained new programmers on the system, well, they didn't have a choice. They had to replace it.

It's adorable that these answers are all in the past tense. This is still going on. Many of these systems still exist.

Re: Ask HN: Was the Y2K crisis real?

#360
Y2k wasn't a bug at first, it was a cost and space saving feature. It only became a bug after the practice never changed until very close to the deadline.

Just a public service announcement that the decisions you make today for good reasons can, in retrospect, be seen as a huge mistake. The future view of your decisions will always have better knowledge than you have when you make that decision.

Post reply on HN