Live data from Hacker News

Due to failure in the IT system, it is not possible to run any trains today

ns.nl

61–70 of 365 posts

Re: Due to failure in the IT system, it is not possible to run any trains today

#62
post #14

My curiosity is killing me as to what exactly went wrong. And as someone living in The Netherlands I'm also kind of mad at the apparent fragility of this huge chunk of our infrastructure. If this had happened on a work day, it would have been a real national emergency, rather than just a huge national inconvenience. I would appreciate one of those post-outage "what happened" reports like with the AWS and Facebook out…

From the notice:

> It affected the system that generates up-to-date schedules for trains and staff.

...boy oh boy did trying to look into what NS uses for crew scheduling ever send me down a rabbit hole.

I don't know if the systems have changed but while poking around online I found this[1] doc about the Netherlands' timetable revamp around 2006 and it talks about the complexity of TURNI-- their on-the-fly crew scheduling system.

> A typical workday at NS includes approximately 15,000 trips for drivers and 18,000 for conductors. The resulting number of duties is approximately 1,000 for drivers and 1,300 for conductors. This leads to extremely difficult crew scheduling instances. Nevertheless, because of the highly sophisticated applied algorithms, TURNI solves these cases in 24hours of computing time on a personal computer. Therefore, we can construct all crew schedules for all days of the week within just a few days.

Then I found more detail about TURNI's implementation in this[2] paper about optimizing crew scheduling for timetables.

> In the railway industry the sizes of the crew scheduling instances are, in general, a magnitude larger than in the airline industry. Moreover, crew can be relieved during the drive of a train resulting in much more trips per duty than typical in airlines. In other words, the combinatorial explosion is much higher. The latter has made the application of these models in the railway industry prohibitive until recently.

Cool stuff.

Finally, gleaning from ns.nl's careers page[3] everything else in their IT land outside this system runs off SAP (likely including the actual distribution of the output of crew scheduling) so if I had to gamble I'd say the failure happened somewhere in the integration between them.

[1] https://homepages.cwi.nl/~lex/files/Interfaces.pdf

[2] https://repub.eur.nl/pub/11701/ei200803.pdf

[3] https://werkenbijns.nl/werkgebieden/it/sap-specialist-bij-ns...

sidenote: If anyone out there is an SAP specialist ns.nl looks like a pretty great place to work: 36hr week, 5 weeks vacation, pension, and free unlimited 2nd class + low cost 1st class train travel.

Re: Due to failure in the IT system, it is not possible to run any trains today

#63

Earlier quoted context omitted.

Only in Amsterdam inside Netherlands after waiting a long line. At the same time Hungarians in Romania could vote by mail and the government was using Fidesz to collect the votes and threw out votes of the opposition. The soft dictatorship works very well, Hungary will go even more down in the future. I'll probably leave Hungary before Hungary leaves EU/NATO and gets into a war.

Voting by mail is insecure? This is very troubling.

It's not simply voting by mail, it's voting by mail by not even using official the post office in Romania.

I know that you're probably refering to US, but I don't make any comment about it, because I don't live there, so I don't know what really happened in US.

In Romania some people found partially burned ballots with only opposition votes next to a garbage bin before the elections ended. The proofs of voter fraud couldn't have been clearer. It's supposed to be a scandal, but nothing will happen about it.

https://hungarytoday.hu/discarded-burnt-postal-ballots-scand...

Re: Due to failure in the IT system, it is not possible to run any trains today

#66
post #7

Earlier quoted context omitted.

I think that trains can travel in both directions on the same track and the schedule keeps them from hitting each other head-on.

What if they are late?

Here is a theoretical solution:

Lets say there is a north-south train line that is shared.

The schedule can create a number of north slots and south slots into which a train of a given speed and length can fit. A variety of different trains could fit into the same slot.

If a train is late, that train is moved into a later slot. That leaves a free slot where the train was going to go. Perhaps every later train is then delayed into a later slot also. Lets call this a rippling delay.

You would probably schedule some spare slots to prevent delays rippling through the entire day's schedule. Some slots might be long enough to permit several trains going in the same direction to travel (it's more efficient if 10 north trains go then 10 south trains rather than 10 north/south interleaved).

Re: Due to failure in the IT system, it is not possible to run any trains today

#67
post #42

In 2019 there was a power blackout in London. When power was restored, a number of new trains refused to start without a technician physically coming over with a diagnostics laptop to them because power fluctuated outside allowed bounds or something before the blackout.

It's likely the same power fault could occur in a single train in other circumstances. When an electric train cannot access the grid reliably it would become far more dangerous - no more regenerating brakes, no reliable acceleration ...

Better to force the operators to fix the boom arm / whatever other cause of power glitches with fault codes like this. Even if they false-trip when there are grid level outages.

Re: Due to failure in the IT system, it is not possible to run any trains today

#68
post #16
post #7

Earlier quoted context omitted.

I think that trains can travel in both directions on the same track and the schedule keeps them from hitting each other head-on.

scheduling hasn't been used for preventing conflicting movements since the 1830s https://en.wikipedia.org/wiki/Railway_signalling#History_of_...

It still sometimes is. There was a big crash in Czech republic recently, where two passenger trains (motor units) collided head on between stations.

Some local routes with few trains a day, where there is no longer any track-side staff, operate in a simplified rules (there is a whole code for that). There, the train will arrive to a station and depending on a symbol in the timetable, it will either continue to the next station without any signals or permissions needed or will wait for crossing train in oposite direction (the train driver must also unlock the station office and call the dispatcher on phone in some cases). Switches are either operated by train crew or they are self-returning (one direction always go to track 1 in station, the other direction goes to track 2, and the switches can be travelled through in the oposite direction without setting them first (this is an accident on standard switches) and they return back.

So, the problem was that the crossing was there on some days and wasn't on some other days, driver forgot or misread the day and the train left into oncoming train...

Was a big discussion after that to do anything, even an uncertified mobile app to shout "train coming" at the driver... Otherwise Czech rep. has a very modern European rail infra, just some local branch routes run like this saving them from being cancelled completely, but the rules were simified maybe too much.

Re: Due to failure in the IT system, it is not possible to run any trains today

#70
post #62
post #14

My curiosity is killing me as to what exactly went wrong. And as someone living in The Netherlands I'm also kind of mad at the apparent fragility of this huge chunk of our infrastructure. If this had happened on a work day, it would have been a real national emergency, rather than just a huge national inconvenience. I would appreciate one of those post-outage "what happened" reports like with the AWS and Facebook out…

From the notice: > It affected the system that generates up-to-date schedules for trains and staff. ...boy oh boy did trying to look into what NS uses for crew scheduling ever send me down a rabbit hole. I don't know if the systems have changed but while poking around online I found this[1] doc about the Netherlands' timetable revamp around 2006 and it talks about the complexity of TURNI-- their on-the-fly crew sched…

[deleted]
Post reply on HN