Computers are an oppressive technology

A human loan manager, given only a credit score, will approve the loan based on only the credit score. The problem isn't human vs machine, it's context-awareness and context-sensitivity vs a lack of one or both.

All large systems, human or machine, tend to be implemented using abstractions. Layers of context-stripping abstractions were a part of buerocracy far before machines took it over. If a loan request form doesn't have an "if I don't get this loan my family will literally starve to death" box that you can check, the evaluator will not know about that context couldn't possibly act based on it. The form can be digital or paper and the evaluator can be a program or a human - it doesn't change a thing.

Similarly, context-sensitivity (the willingness to let the context sway your decision) was strictly limited even back in the day when decision makers were close enough to the source to be aware of the context. A bank clerk had to stick to the rules and couldn't just decide to give a loan to someone that didn't qualify under them. If they did, they'd be risking losing their job or worse, so they didn't. This discretion was only given to the people in charge - and that hasn't changed with computers. A person in a high enough position can override what the computer says, just as easily as they can override a lower clerk's decision.

Yes, making machines context-aware takes some effort, and making them context-sensitive takes even more effort. But even if machines magically came with both, their operators would do the best to get rod of them, just like they did with their employees. From their point of view, empathy is inefficiency. And that's the real problem.

It is also worth noting that those examples have a positive element as well. Train operators are too far from the doors to know when it is safe to depart. The closing of the train door ensures everyone is safely inside before the train can depart. Having a human operator who uses discretion at each door would only create a new set of problems, namely of train delays causing the entire system to grind to a halt (at best) and safety issues (at worse).

Likewise for the food dispenser guillotine. While the case described sounds poorly designed, the general idea behind those doors is to prevent thieves from leeching upon unsupervised vending machines. One may argue that a human could use discretion for when a person is truly starving. On the other hand, that after hours supply of junk food wouldn't exist without vending machines since the labour involved would make it too expensive. It is also worth noting that a real human would rarely hand out food that is not their food to give away.

As a side note: I also detest the use of the term jobsworth to describe something the author doesn't agree with. More often than not, those rules exist for a reason. While preventing the submission of a form a second late is not one of them, encouraging the submission of the form in a timely manner is. There are many people behind the scenes who need to get their job done for the benefit of those who filled out those forms in a timely manner.

I disagree that "the system" is "meant" for anything - the system is not a monolith even if certain parts of the machinery work well together, it is rarely a question of design, usually a mishmash of various historical phenomena and trends that happen to exist simultaneously.

I think my plea for "a little bit of this and a little bit of that" perhaps got somewhat muted in my comment, placed at the very end like it was. So I agree that it would be oppressive to always treat everyone equal, the billionaire as well as the double working lone mom, and that machines many time is a prime tool for oppression and deflecting responsibility.

But it would be problematic to start chipping away at the ideal of everyone's equality under the law in order to rectify such. This ideal is what stands between institutions like freedom, democracy and human rights on the one hand and corrupt states on the other, where the law is applied differently depending on who is on trial, and where the letter and intent of the law is flaunted in favor of pure (real) oppression.

The problem really seems to be discerning where to apply "a little bit of this" vs where to apply "a little bit of that", a project that doesn't lend itself well to sweeping generalizations.

The point of the article boils down to something like this:

Pre-AI computers are very literal and show no judgement or discernment in their rule-following. They are inevitably used to implement complex systems against which users have little recourse, and to implement complex systems whose behaviours and failures can be blamed on computers rather than on the people responsible. These fundamental problems will persist no matter the progress made in AI.

Free Software is intended to stop the user being oppressed by their own computers. The article's point is about other people's computers being used to serve their interests over yours, in ways that may be detrimental to the user compared to a more 'manual' system.

Consider the train door example from the article. Suppose that code were Free Software. This might benefit whoever owns and runs the train, but wouldn't benefit the user (of the train) at all.