Shouldn't the defense attorney consider the possibility that computers are not working correctly? Is there... not a defense attorney?
The presumption that computers are working correctly
131–140 of 171 posts
Re: The presumption that computers are working correctly
#132After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.
Sadly this often happens because often times the machine works 99% of the time. So when the thing starts failing in 1% of the cases people will dismiss it because it worked in the previous 99% of the time. People have trained themselves to trust the results so questioning it does not even enter their minds.
It does remind me though of the book Systemantics: https://www.amazon.com/Systemantics-Systems-Work-Especially-...
One of the core thesis in the book is :
The system itself does not actually do what it says it is doing (The Operational Fallacy).
This people and engineers as well often forget. Just because the system says it does X does not mean it actually does it. People would do well to cultivate a healthy skepticism towards those complex IT systems. They might not fail most of the time but they inevitably will.
Re: The presumption that computers are working correctly
#133Earlier quoted context omitted.
I agree, I find it hard to read about because it makes me so angry. Four people driven to suicide, hundreds of lives ruined. It took them so long to hold all the inquiries and decide on compensation that dozens have died before being acquitted or receiving compensation.
And that is only the one we know of. There are thousands of cases that are hidden in the bowels of the post office court system Oh and the compensation are uh... let say hard to get the money for.
And I've yet to hear mention of the people who weren't prosecuted because they paid the excess out of their own pocket. From the stories I have read, it seems that many who were prosecuted had tried that but ran out of money. Many more must have successfully dealt with the "shortfall" and are surely due their own restitution now?
Every transaction the system touched should be inspected, because I'm sure there were plenty of sub-postmasters who took the "shortfall" at face value and paid up without even thinking about questioning the system.
Re: The presumption that computers are working correctly
#134Earlier quoted context omitted.
What makes you think this isn't just a software bug?
Because "too hot to handle" tells me that this should have really been prevented at the hardware level, and this is also common practice.
Power control is all configuration. Hardware merely provider the signals and mechanisms for SW control.
Re: The presumption that computers are working correctly
#135Earlier quoted context omitted.
If bugs were easy to find and solve there wouldn't be an entire highly paid industry to do it. Add to that malicious actors actively trying to work against systems (bank/post office/medical system etc) and it becomes pretty hard to prove that both the software and the data referenced by the software is in fact correct. Now think about the last time a critical piece of data was recorded on paper with a pen/analogue ty…
If programs were easy to prove correct, there wouldn't be a testing and debugging industry. Bugs don't have to be root caused and solved in order to call into question the use of some computer situation as circumstantial evidence. The claimed buggy, externally-visible behavior just has to be reproduced. If there are third-party reports of it, that would help.
I would argue programs are impossible to prove correct, I would speculate it is related to the halting problem.
That doesn't mean that in general the subsections of it related to any specific court case isn't correct.
Unfortunately disproving something is much much harder than proving something.
Proving that there was a bug in a program is hard but proving that there isn't is nearly impossible and that is likely why the law is the way it is.
Re: The presumption that computers are working correctly
#136Earlier quoted context omitted.
Because "too hot to handle" tells me that this should have really been prevented at the hardware level, and this is also common practice.
Power control is all configuration. Hardware merely provider the signals and mechanisms for SW control.
Within limits, yes.
Re: The presumption that computers are working correctly
#137Earlier quoted context omitted.
What exactly have you tried to do there? Isn't a schema already a model? What properties did you want your model to have? Yes, in general it is difficult, because then software becomes mathematics. And that's on top of the software doing something useful and what the user wants. But I think we are now getting the tools to get there. You will still need to become a mathematician, though.
> What exactly have you tried to do there? Isn't a schema already a model? What properties did you want your model to have? Yes, a schema is a data-model, and a database can tell you whether the row you are trying to insert satisfies the constraints, but it can't tell you whether an instantiation of the database exists, nor can it tell you how to construct it. There are 2 issues here, first proving that an instantiat…
Ok, I understand. So first you want to see your schema as a property on a database (which is just a bunch of tables with data, I guess), and see if there is actually any such database, or if the property is always false. Second, assuming that there are any such databases: Given certain commands which operate on databases, is there any sequence of commands which transform the empty database into a database such that the schema property holds for it? Actually, you probably would want the property to be true for every intermediate step as well, right?
Sounds interesting! Although in practice, solving this is probably not that important, because if you cannot come up with some examples of constructing databases for your schema and application, then the schema is the wrong one pretty much by definition.
Re: The presumption that computers are working correctly
#138This sort of thing has been going on since ... well, forever. I was reminded of this post [1] by Phil Agre in the RISKS Digest from 1988:
"I've noticed that responses to attempts to complain about uncorrected technical problems tend to take certain recurring forms ...
What has happened? Someone, in this case a customer though in other cases it has often been an employee, complains that something is badly amiss with some system, receives no redress, and finds that their reports are ignored. So they make noise about it. The response?
1. Misplaced accusations of uncooperativeness. ...
2. Condescending lectures about how `incredibly complicated' the systems in question are ... "
etc. See the post itself for much more:
Re: The presumption that computers are working correctly
#139Earlier quoted context omitted.
People outsource their critical thinking all the time, but with computers the failures are more egregious. There is less recourse to correct things, and information propagating through systems can make the error more persistent. Also: people take the verdict from a computer as a given even when reality and computer disagree. I've used this story a couple of times before on HN, and if it hadn't happened to mother's co…
I had a similar experience as your moms colleague. I transferred 20k from savings to checking. During the process I think what happened is the web banking page froze and allowed me to hold the enter key on confirm that repeated several times. I ended up with 200k in my account that came as a negative 180k balance to my saving account sent to my checking account. (values rounded for story simplicity) This meant that n…
Here is a posting[1] from a thread about a somewhat related experience with a bank more than 30 years ago, from the RISKS Digest, 1988:
"Someone, in this case a customer though in other cases it has often been an employee, complains that something is badly amiss with some system, receives no redress, and finds that their reports are ignored. ..."
Re: The presumption that computers are working correctly
#140How does that work? I'm not a lawyer. I'm a lazy programmer. So, does the defense get to subpoena the logs, and depose a programmer to explain why some random request returned a 502? Does the defense get, like, red team access? Although I'm a lazy programmer, I do think a lot about failure modes. If my freedom is on the line, I can probably think up a few (orders of magnitude) test cases. Happy to provide a shell scr…
The real problem is this isn't a "right" that the postmasters could exercise. You're accused of stealing money. You're not paid very much in the first place and have only limited legal aid, and may even be in prison (on remand) already. There's no "defense team", you only have a solicitor assigned to you at the police station. You're also not a programmer. How are you going to argue that the database which shows you…
And the only evidence the prosecution needs to show is some log messages from the system saying the math doesn't add up? Nothing else?