Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The presumption that computers are working correctly

emptycity.substack.com

Post reply on HN
Print view View original

61–70 of 171 posts

Re: The presumption that computers are working correctly

#61

Post by uoaei »

vasco wrote: post #45
bmitc wrote: post #40

Earlier quoted context omitted.

> I’m often surprised that computer systems work at all. I also remain contunually amazed that anything works. It boggles my mind, because I have seen how software gets built. How can it possibly all work? I think my answer to this conundrum and seeming paradox is that it doesn't all work. We have trained ourselves and been trained to accept things that aren't working as something else. And like you say, people treat…

Any sufficiently complex system operates constantly in failure mode

This is an incomplete view of complex systems. It is more accurate to say they operate constantly in recovery mode.

Re: The presumption that computers are working correctly

#62

Post by ozim »

I can see why they changed it. I expect in last 3 decades amount of computer evidence grew up to amounts where there is no practical way to prove computer was operating correctly.

If I present a digital photo as evidence it would be dismissed by defense lawyers to easily. Then basically any digital evidence would be dismissed the same way.

I can see how it was easy to manage in the 80’s and not anymore in 00’s.

Re: The presumption that computers are working correctly

#63

Post by hulitu »

bboygravity wrote: post #52
zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

I would add some nuiance and state that a lot of popular software is broken to the point that it's main function (the reason that the software exists) is not working. Some examples: -- The websites of various airlines (TAP, Vueling, KLM) have often had bugs in them (in the past 2 years when I used them) to the point that they don't allow you to book tickets or log-in with your account. I would say they where broken m…

>Teams suddenly reversing the order of the letters in your sentence as you're typing in a chat

I think they had a button for this on the right click menu.

TBH, Teams is the quintesence of today's SW development state of the art.

Re: The presumption that computers are working correctly

#64

Post by drt5b7j »

phire wrote: post #7

As a programmer, it's my experience that the computer itself usually works correctly. It's almost never a hardware bug. But software.... software gets things wrong all the time and the computer hardware just blindly does what the software told it to.

It's appears to you this way because you're an insider in software. If you were in hardware, you would believe the opposite.

Re: The presumption that computers are working correctly

#65

Post by bostik »

stevesimmons wrote: post #53

David Allen Green is a great commentator on law in the UK. During the Brexit period, he was a contributing editor to the Financial Times, with similarly insightful analyses of how Boris Johnson et al were misusing the government's legal powers. Anyway, David also posted this to his non-gated "The Law and Policy Blog" at [1]. [1] https://davidallengreen.com/2023/09/computer-says-guilty-an-... There are more comments t…

Thank you, I have been looking for a resource like this, one that would come with an RSS feed. Now I have one less thing to hunt around for.

Re: The presumption that computers are working correctly

#66

Post by cf141q5325 »

di4na wrote: post #18
suzzer99 wrote: post #12

This is the first I've heard of this scandal. I'm curious what the major problems were with the software that caused the discrepancies. Dropped connections? Wikipedia didn't go into much detail with that. Ok I found this: https://www.benthamsgaze.org/2021/07/15/what-went-wrong-with...

It is one of the biggest scandal of our time related to software. I ... want to recommend learning more about it but only do so with a strong stomach. Numerous people that rabbithole into this case end up running to a nearby bucket. It is... not pretty.

Its a major problem with many of such issues. Some stuff is just really horrible, often even without any clear approach of how to fix it. It then gets even more problematic if people react to this by retreating into easy stories that act as mental fig leaves. Especially once these leads to clearly dysfunctional attempts to solve the issue, making the situation even worse for those effected.

Re: The presumption that computers are working correctly

#67

Post by denton-scratch »

masfuerte wrote: post #59
denton-scratch wrote: post #51

A big part of the problem with these prosecutions is that the prosecuting authority is the Post Office itself. Until this case came along, I had no idea that the Post Office had these powers. So effectively, the Post Office was acting as both complainant and prosecuting authority. "The Post Office Investigation Branch (IB) is the oldest recognised criminal investigations force in the world. For more than 335 years it…

Anyone can bring a prosecution in England. In theory it provides a way to hold the powerful to account but it's very expensive so almost all private prosecutions are actually by corporations.

It's true that anyone can bring a prosecution. It's also true that the Crown Prosecution Service can take over a private prosecution, and then drop the case.

Obviously, the CPS should have done this in the Post Office case. During the relevant period, the CPS was underresourced and overloaded (it still is now).

Re: The presumption that computers are working correctly

#68

Post by auggierose »

gwd wrote: post #48
murderfs wrote: post #20

Earlier quoted context omitted.

As a programmer that works on drivers, it's my experience that hardware almost never works correctly. "We'll fix it in software" might as well be the official motto of the hardware industry.

But thinking about a court case, what should be the presumption? Should it be up to the prosecution to prove that the RAID controller / network card didn't corrupt the data or that the GPU did the math correctly? Or should it be up to the defense to prove that it didn't? And how would each side demonstrate their case to a reasonable level of certainty? As others in this thread have said, for bits of the system that h…

Mathematical verification of both hardware and software should be mandatory, at least as long you rely on it for the law. Simple as that. For example, how can you trust a photo, when tons of computational photography has been applied to it, and you don't know the exact amount the algorithms can change the photo?

Re: The presumption that computers are working correctly

#69

Post by bmacho »

vasco wrote: post #45
bmitc wrote: post #40

Earlier quoted context omitted.

> I’m often surprised that computer systems work at all. I also remain contunually amazed that anything works. It boggles my mind, because I have seen how software gets built. How can it possibly all work? I think my answer to this conundrum and seeming paradox is that it doesn't all work. We have trained ourselves and been trained to accept things that aren't working as something else. And like you say, people treat…

Any sufficiently complex system operates constantly in failure mode

> The Fundamental Failure-Mode Theorem (F.F.T.): complex systems usually operate in a failure mode.

https://en.wikipedia.org/wiki/Systemantics

now this is depressing. I hate everything and I live in a constant 'everything is shit' state of mind, but this book hates things way more than me.

Re: The presumption that computers are working correctly

#70

Post by rwmj »

jfoutz wrote: post #8

How does that work? I'm not a lawyer. I'm a lazy programmer. So, does the defense get to subpoena the logs, and depose a programmer to explain why some random request returned a 502? Does the defense get, like, red team access? Although I'm a lazy programmer, I do think a lot about failure modes. If my freedom is on the line, I can probably think up a few (orders of magnitude) test cases. Happy to provide a shell scr…

The real problem is this isn't a "right" that the postmasters could exercise. You're accused of stealing money. You're not paid very much in the first place and have only limited legal aid, and may even be in prison (on remand) already. There's no "defense team", you only have a solicitor assigned to you at the police station. You're also not a programmer. How are you going to argue that the database which shows you stole money wasn't working correctly?
Post reply on HN