Computers are an oppressive technology

This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: government bureaucracies, beverage dispensers. They own the computers and impose their own rules on us, rules like deadlines and brand checking. We don't want these rules but they impose it on us all the same.

Computers are neutral technology. It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. The computers belong to them now, they're just letting us use them. We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them: radiofrequency hardware firmware, digital rights management, encryption regulation. If left unchecked, the power of computing will cause massive damage to the entire intellectual property industry as well as many government functions such as law enforcement. Computing freedom is therefore against their agendas: they want to transform computers into tools of oppression in order to preserve the status quo or change it in their favor. We can buy iPhones but they're not really ours, they still belong to Apple, the software is under Apple's control so when they suddenly decide to oppress us with client side scanning there's nothing we can do.

Would you seriously like to go back 150 years and see how that feels?

Because of "SOX" I run our "exit process" and I had a small victory lately. I'm not proud, but it happened, and I was thankful.

Computers are fine but total computerisation of human society may not be so. That's actually what the article should have explored in my opinion. Computers are neither ruthless nor kind, its controllers, humans, are. So the question then becomes, who controls them and how sophisticated they (the computers) become, and how much they infiltrate our world. A simple thought... should everything that can theoretically be controlled by computer algorithms, made so? Should computers use ML for as many as tasks as possible, even though that can lead to wrong decisions on occasion?

I'm not sure that's exactly true and I'm not sure on it's applicability to the article.

The author's point isn't about that it disregards human feelings in my opinion, it's that we implicitly trust and idolize the machines without thinking about the biases and uses for a lot of the technology we rely on. I actually feel the examples used by the author are not great and a bit showy and repeating their initial anecdote, but let's consider some technologies that carry the same premises the author comments on but have a lot of misuses by humans.

- KPI tracking tools. Useful of course, but what a lot of persons miss is the I in KPI, which is __indicator__. Numbers don't lie, but the people reading the numbers do, often unintentionally. Consider the classic example of a company struggling with customer satisfaction (CSAT) that looks just at KPIs, sees the entire team is in the green save CSAT, but customer satisfaction is getting worse and worse. The management is scratching their heads because the numbers seem contradictory, but in fact, they fail to realize that the KPI for closed cases has been optimized for by the support team and emphasizes closing cases by any means instead of solving issues correctly

- Human detection mechanisms that fail to detect persons with darker skin tones (this was frequently an issue even as recently as the 2010's when the tech was getting a ton of attention); the researchers didn't consider or didn't bother to test with darker skin tones

- Speech to text that cannot handle accents/dialects or doesn't work for non-english at all (I even found this today talking with a colleague who is a non-native English speaker; as I gushed over the M1 Mac dictation tool being pretty damn good, she reminded me it's because I'm a native speaker and the dictation is not very good at handling non-native speakers. Their accents may be parsable by humans, but the dictation AI is not trained to really handle this)

- Edit: adding this one cause it hits close to home. Call time monitoring that punishes non-native speakers or persons with speech impediments because they need to ask additional questions or just take longer to convey the same point. At one of my workplaces where I oversaw support, I lost a number of good engineers to management KPI obsession because of such non-sense. They were demonstrably better in every technical metric than their English-native speaking peers, but the KPIs for calls showed them to be inefficient. Rather than analyze the actual content and capabilities of the engineers, when layoffs came, the KPI was the only metric considered and we lost some very good tech people needlessly. I did not stay with this organization for long as I was disgusted by such practices and my concerns fell on deaf ears.

The list can go on, and yes, implicitly the machines are working exactly as designed and there are human failings that go into the misinterpretations. The problem is the machines being hailed as the ultimately objective and functional implementation, an authoritative statement on the condition the machine is programmed to check, and this is exactly how they're marketed or even thought of by the creators.

Add in the continuing obsession with minimum viable products to get to market and you just end up with these implicit biases being baked into products from the beginning and it can have very nasty side effects.

That is the concern I believe from the article; it's not just that machines do exactly what they're programmed to sometimes bad outcomes, but that we put a lot of faith and trust into these machines and models without questioning it.

I respond to this post as I think it's not a safe thing to assume that when you program you try to fulfill the user wishes as much as possible -- WebDev is a perfect example filled with tons of "Dev-friendly" features that users loathe, but it's better for the Dev and it looks better for the company to have these. Or even just talk with your colleagues from time to time and ask how many shortcuts they've taken that is a pain for the users, but ultimately easier to write and lets you ship sooner. Hell, I'm guilty of this so I'm not without sin.

My takeaway from the article is that we need to avoid the idealized understanding of programming and avoid concepts like "Code is Law", because it fails to recognize the biases we write into our code and systems. Don't take biases as political either, think about simple use cases you miss and never consider when writing some program or modeling your data structure, innocently enough in most cases, but now it's baked into the program because refactoring is too expensive due to tech debt.

A program and a hungry/scared mountain lion might indeed both just be acting as they're programmed; no one is trying to convince me to trust the mountain lion though.

Institutions made of people have limited reach. Human resources are finite, there's a physical limit on the scale of what they can do. A computerized society is limitless. A state might want to spy on everybody but it's impossible to do so due to limited manpower. Computers remove those limits and allow states to implement global surveillance.