Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The presumption that computers are working correctly

emptycity.substack.com

Post reply on HN
Print view View original

51–60 of 171 posts

Re: The presumption that computers are working correctly

#51

Post by denton-scratch »

A big part of the problem with these prosecutions is that the prosecuting authority is the Post Office itself. Until this case came along, I had no idea that the Post Office had these powers.

So effectively, the Post Office was acting as both complainant and prosecuting authority.

"The Post Office Investigation Branch (IB) is the oldest recognised criminal investigations force in the world. For more than 335 years it has worked to detect offences against the post and prosecute the perpetrators of these crimes."

https://www.postalmuseum.org/blog/the-post-office-investigat...

Re: The presumption that computers are working correctly

#52

Post by bboygravity »

zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

I would add some nuiance and state that a lot of popular software is broken to the point that it's main function (the reason that the software exists) is not working.

Some examples: -- The websites of various airlines (TAP, Vueling, KLM) have often had bugs in them (in the past 2 years when I used them) to the point that they don't allow you to book tickets or log-in with your account. I would say they where broken more often than not. -- Instagram and Google have been deactivating people's accounts (automatically) for no apparent reason. I've had this happen with Instagram 3x in the space of 2 months after which I just gave up and stopped using it. -- Skype, no explanation needed for those who use it. -- Teams fails in many amazing ways no matter how often they "update" it. I've heard stories about the mute button not actually muting, Teams suddenly reversing the order of the letters in your sentence as you're typing in a chat, it just shuts down in the middle of a call, unresponsive, etc etc etc. The amount of productivity these bugs in this software burn worldwide must be significant and the real cost to companies in the form of wasted man-hours is likely higher than whatever they pay in licensing fees to Microsoft. -- Telegram can't handle an external monitor (the scaling gets all messed up causing you to have to restart Telegram to fix it). -- iCloud takes between seconds and an hour or so to sync files even when all devices are on a good connection.

Re: The presumption that computers are working correctly

#53

Post by stevesimmons »

David Allen Green is a great commentator on law in the UK. During the Brexit period, he was a contributing editor to the Financial Times, with similarly insightful analyses of how Boris Johnson et al were misusing the government's legal powers.

Anyway, David also posted this to his non-gated "The Law and Policy Blog" at [1].

[1] https://davidallengreen.com/2023/09/computer-says-guilty-an-...

There are more comments there, including this one from me (written for his blog's legal/non-technical audience, not for the HN crowd!):

"There has been a big shift in the practice of computer programming in the last 25 years. In the “old days”, software was delivered on floppy disks or tape, and pretty much had to be correct first time.

Nowadays, everything is on the web, “agile” means the functionality gets delivered incrementally in 2-week sprints. It is regarded as ok to leave out the difficult cases or ship bugs, provided they don’t affect too many users too much of the time. In other words, the software is never really finished.

With that in mind, it becomes simple to rebut the presumption of correctness: if the company’s tech organisation describes itself as “agile”, using “scrum” etc, its software – by design – contains bugs and omissions, both known and unknown.

Fortunately, modern development practices means evidence for this will be in the dev tickets in GitHub, Azure DevOps or whatever other systems their dev teams use."

Re: The presumption that computers are working correctly

#54

Post by di4na »

zelos wrote: post #31
di4na wrote: post #18

Earlier quoted context omitted.

It is one of the biggest scandal of our time related to software. I ... want to recommend learning more about it but only do so with a strong stomach. Numerous people that rabbithole into this case end up running to a nearby bucket. It is... not pretty.

I agree, I find it hard to read about because it makes me so angry. Four people driven to suicide, hundreds of lives ruined. It took them so long to hold all the inquiries and decide on compensation that dozens have died before being acquitted or receiving compensation.

And that is only the one we know of.

There are thousands of cases that are hidden in the bowels of the post office court system

Oh and the compensation are uh... let say hard to get the money for.

Re: The presumption that computers are working correctly

#55

Post by di4na »

lozenge wrote: post #47
suzzer99 wrote: post #12

This is the first I've heard of this scandal. I'm curious what the major problems were with the software that caused the discrepancies. Dropped connections? Wikipedia didn't go into much detail with that. Ok I found this: https://www.benthamsgaze.org/2021/07/15/what-went-wrong-with...

I think the issue wasn't the bug but the lack of care? Pretty much any large system develops bugs during the development but they get noticed and fixed. These bugs got noticed and not fixed.

Worse. They knew of the bugs and prosecuted postmaster for the discrepancy instead of accepting them as bugs.

Re: The presumption that computers are working correctly

#56

Post by didntcheck »

The last time I looked into the Post Office scandal, the impression I got was that "we just had too much faith in the IT system!" was a convenient story, but the high-ups knew full well about the problems, and full well that they were persecuting innocent people, but did it anyway to cover up their own failings. If so, people need to see substantial jail terms for this. Perverting the course of justice is no misdemeanor. There's even allegations that Paula Vennells knowingly lied to Parliament

Re: The presumption that computers are working correctly

#57

Post by jpc0 »

zubairq wrote: post #43
kazinator wrote: post #27

Every prosecutor in possession computer evidence of a crime having to prove the software correct is a nonstarter. Assuming it's right, and proving it buggy, is the right principle.

I’m quite interested to learn and understand the line of reasoning for this comment. Care to elaborate?

If bugs were easy to find and solve there wouldn't be an entire highly paid industry to do it.

Add to that malicious actors actively trying to work against systems (bank/post office/medical system etc) and it becomes pretty hard to prove that both the software and the data referenced by the software is in fact correct.

Now think about the last time a critical piece of data was recorded on paper with a pen/analogue typewriter and both parties got a physical copy of it.

Last time you made a cash withdrawal did you have to go to a teller at the bank, fill in paperwork, have two copies of that paperwork exist and then that paperwork was digitised or was the entire process digital beginning to end?

Think about how you, if you were an expert witness, would prove that an transaction made on a banking website was not a computer fault.

Do you show a code audit which meets some standards?

Do you have the bank's CISO explain all of the security measures?

INAL but I'm reasonably sure the defendant only has to prove that a system with the same security and audit trail has in fact been faulty and your evidence goes out the door or at least the validity of it decreases.

Re: The presumption that computers are working correctly

#58

Post by TeMPOraL »

makeitdouble wrote: post #37
nine_k wrote: post #35

Earlier quoted context omitted.

Given enough eyeballs, all bugs are shallow. As such, they are complained about by thousands of people, for years. Less ironically, one can expect widely used features of widely used software to have been tested in realistic conditions so whatever bugs there might be are at least known. But it takes an insane amount of testing to cover the phase space even of a moderately complex piece of software: look how large is…

Yes. Bugs are there, but either they're shallow, well known and worked around, or have low impact for the majority of people using them. The epitome of that was utf-8 support for a long time: there were a huge number of bugs, but alaphabet based languages would be mostly ok, and most devs would know it was a mess and work around that as much as they can. To me one should expect any software to be broken in numerous w…

> That looks awfully close to the micro-service approach as well. The approach can be good, but it still gives me pause.

Microservices aside, GP's statement seems to imply composition is like addition wrt. complexity. Unfortunately, in reality it's multiplication. Complexity multiplies with each moving part. There's hardly a better example of that to what we inherited from "original Unix" - chaining small, focused tools in pipes turns into a write-only mess after around the fourth pipe.

Software complexity isn't simply a divide-and-conquer problem, because we pay a penalty for each individual piece, and each piece of glue that connects it with other pieces. That's the whole reason we rely on abstraction and generalization so much - the ability to take a bunch of "small, self-contained, isolated, well-understood parts", put them in a box, and stick a higher-level name to it. Recursively.

Re: The presumption that computers are working correctly

#59

Post by masfuerte »

denton-scratch wrote: post #51

A big part of the problem with these prosecutions is that the prosecuting authority is the Post Office itself. Until this case came along, I had no idea that the Post Office had these powers. So effectively, the Post Office was acting as both complainant and prosecuting authority. "The Post Office Investigation Branch (IB) is the oldest recognised criminal investigations force in the world. For more than 335 years it…

Anyone can bring a prosecution in England. In theory it provides a way to hold the powerful to account but it's very expensive so almost all private prosecutions are actually by corporations.

Re: The presumption that computers are working correctly

#60

Post by vladms »

zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

Most people have a tendency to take the easy path and think stuff is perfect rather than accept everything works "approximately", although we see again and again proofs of the contrary.

But as an engineer I find this natural and efficient! You should prioritize and some bugs are less important than others. If anything more effort should be put in education, health and fundamental research rather than fixing all bugs in all software.

Post reply on HN