UK court clears post office staff convicted due to ‘corrupt data’ • Hacker News Board

Re: UK court clears post office staff convicted due to ‘corrupt data’

#121

Post by DoubleGlazing » Fri, Apr 23, 2021, 2:27 PM UTC

PaulKeeble wrote: post #35

Software is in the walls. At some point legislators are going to come and ask the question how we stop things like this happening and if the Fujitsu's of the world don't have an answer then we can expect regulation that will likely embed practices that don't help. I don't think we take software reliability seriously enough, most of our focus is on speed of release, ever quicker cycles and it being OK to break things.…

> I don't think we take software reliability seriously enough, most of our focus is on speed of release, ever quicker cycles and it being OK to break things.

This drives me up the walls. At my last job (food ordering startup the CEO had the attitude that releasing code that was 95% functional was Okay, remaining issues could be fixed as we went along.

As a result, one developer overlooked a bug that cost the company €300,000, loyalty discounts weren't being deducted from payments to take-aways. They then had the cheek to demand take-aways pay them back.

Then they launched a major upgrade to the system at 5pm on a Friday - two hours before their busiest time of the week. It collapsed a few hours later and it was impossible to roll back because they didn't include a roll-back SQL script for the DB. It took till the following Tuesday to fix it.

The DB schema was all over the place and as a result it was slow. Entity Framework couldn't handle it and the SQL it was generating was terrible. Me being the only one with decent SQL knowledge had to replace all the bad EF queries with raw inline SQL.

Despite this, they still carried on deploying without a care in the world. I was told to stop moaning about QA. We didn't have QA or testing staff, the CEOs attitude being why pay for QA staff when our clients will do it for free?

Re: UK court clears post office staff convicted due to ‘corrupt data’

#122

Post by lbriner » Fri, Apr 23, 2021, 2:27 PM UTC

PaulKeeble wrote: post #35

Software is in the walls. At some point legislators are going to come and ask the question how we stop things like this happening and if the Fujitsu's of the world don't have an answer then we can expect regulation that will likely embed practices that don't help. I don't think we take software reliability seriously enough, most of our focus is on speed of release, ever quicker cycles and it being OK to break things.…

How do you know this culture existed 20 years ago when this system was developed? It is almost certain that a corporate developer in the 1990s would be 100% waterfall.

The issue is very often related to massively complex corporate requirements (the Post Office makes me cringe, even today, with the complexity of their postal system) and then coupled with the ever-present need to keep costs low, especially when designing something so complex.

I doubt anyone building this thought it would be OK to break things!

Re: UK court clears post office staff convicted due to ‘corrupt data’

#123

Post by toomanybeersies » Fri, Apr 23, 2021, 2:34 PM UTC

gpvos wrote: post #71

> In the latest chapter of one of the biggest miscarriages of justice in English legal history, 39 people who were prosecuted Meanwhile in the Netherlands, ~26000 people have been branded as fraudsters by the tax office due to a way too strict child benefits law. More than 100 probably entirely innocent people fled the country. Even the compensation that is now promised is only slowly trickling towards them, and like…

We had a similar thing happen with unemployment benefits in Australia [1], which arguably led to several suicides.

[1] https://en.wikipedia.org/wiki/Robodebt_scheme

Re: UK court clears post office staff convicted due to ‘corrupt data’

#124

Post by Silhouette » Fri, Apr 23, 2021, 2:34 PM UTC

viraptor wrote: post #117

Silhouette wrote: post #107

Earlier quoted context omitted.

We should have the person presenting any report like that be personally responsible for the contents. If they aren't willing, it shouldn't be presented. I don't think making it personal works at scale. You can't reasonably expect everyone giving evidence in court, say every individual police officer who is a witness to a speeding offence, to be a technical expert on the technological tools they are given to do their…

> say every individual police officer who is a witness to a speeding offence, to be a technical expert on the technological tools they are given to do their job. That's what I was going for. If the officer doesn't understand the limitations of their tool, they shouldn't testify in court beyond "I pointed it that way and read the number, as trained". There are existing cases where the speed reading is contested becaus…

FWIW, I'm reasonably sure that's exactly what does normally happen in that particular case. Police officers sometimes speak in a slightly stilted way in court here in the UK, partly because they use words carefully chosen to be statements of fact as they know them and not to draw conclusions that are a matter for the court to decide.

Re: UK court clears post office staff convicted due to ‘corrupt data’

#125

Post by ClumsyPilot » Fri, Apr 23, 2021, 2:38 PM UTC

neolefty wrote: post #119

marcinzm wrote: post #48

Earlier quoted context omitted.

This seems more an issue of bureaucratic incentives than software. Fujitsu wanted to hide bugs to look better for future contracts. The Post Office wanted to hide bugs to deflect blame from central leadership and be able to scapegoat people at will. The judicial systems seems to have either not cared or had incentives for some quick prosecutions. Software doesn't exist in a vacuum and software will never be perfect.…

Yes, it seems clear that people knew about these problems — they were obvious at one level of management — and they worked together to cover them up.

Heads have to start rolling for this, or we will end up in a dystopian nightmare where any corporate organisation can ruin your life for no reason.

It will be like USSR except more unpredictable because it can come from any direction

Re: UK court clears post office staff convicted due to ‘corrupt data’

#126

Post by Rexxar » Fri, Apr 23, 2021, 2:41 PM UTC

mnw21cam wrote: post #40

PaulKeeble wrote: post #35

Software is in the walls. At some point legislators are going to come and ask the question how we stop things like this happening and if the Fujitsu's of the world don't have an answer then we can expect regulation that will likely embed practices that don't help. I don't think we take software reliability seriously enough, most of our focus is on speed of release, ever quicker cycles and it being OK to break things.…

I still think https://xkcd.com/2030/ has to be taken seriously. You can put a whole load of verification effort into your software, which will undoubtedly make it more reliable. But you are still likely to have some kind of corner case where it breaks down. Software is complex enough for this to be universally true. The key is how we respond when the software fails. The https://en.wikipedia.org/wiki/Therac-25 case sh…

I agree with the sentiment but the example taken for software in this xkcd is wrong. There is a fundamental problem of trust when using software for voting systems that is not linked to the reliability of software but to the nature of voting systems and the properties we want.

Re: UK court clears post office staff convicted due to ‘corrupt data’

#127

Post by londons_explore » Fri, Apr 23, 2021, 2:42 PM UTC

Presumably to put someone in prison for being a money thief, one would need to prove where that money went...

Were all these people accused of theft with not a single record of the yachts they bought with all the money they supposedly stole?

I would assume most of these people would be able to turn over a complete financial record of their lives (ie. I was paid £x, I paid taxes of £y, and here is a bank statement showing how I spent it, and here is whats leftover). How exactly can you imprison someone for theft of money if they can present that?

Re: UK court clears post office staff convicted due to ‘corrupt data’

#128

Post by jjk166 » Fri, Apr 23, 2021, 2:43 PM UTC

lbriner wrote: post #115

davidhyde wrote: post #98

Those postmasters defending charges of theft against them in the 2000s should not have had to prove that the computer system they were forced to use had bugs. In order to prosecute them in the first place, the Post Office should have had to prove, beyond a shadow of a doubt and without risk of bias, that the computer system was correct. So, independent review at the very least, not testimony from parties with a veste…

Unfortunately, that sounds straight-forward but isn't: Q) Did you or any of the people you got to examine the software found any way that what the defendents said was true? A) No Q) Then you are guilty beyond reasonable doubt. I think the bigger issue here is around the power that a large organisation wields to duck and dive and use corporate tricks to manipulate how it played out. For example, the fact that so many…

No, that's still presumption of guilt.

If you're going to convict someone of stealing £59,000, the very first thing you should have to show is that £59,000 actually got stolen. If there is reasonable doubt that the crime took place, no one can be guilty beyond that reasonable doubt. If the defendant claims the computer system got it wrong, it's not enough to say you are unaware of bugs, the prosecution should have to show that the computer's output was consistent with the results of doing the calculation by another method.

Re: UK court clears post office staff convicted due to ‘corrupt data’

#129

Post by gerjomarty » Fri, Apr 23, 2021, 2:46 PM UTC

whyleyc wrote: post #106

There's a great 10 episode Podcast on this debacle on BBC Sounds: https://www.bbc.co.uk/sounds/series/m000jf7j It's really well paced and includes contributions from many of the sub-postmasters affected by this scandal.

I hadn't heard about the story until the BBC started re-running this series this week. Absolutely shocking that flaws in the system were dismissed and suspicion thrown on the sub-postmasters instead.

Re: UK court clears post office staff convicted due to ‘corrupt data’

#130

Post by rlpb » Fri, Apr 23, 2021, 2:49 PM UTC

lbriner wrote: post #118

haunter wrote: post #43

https://www.computerweekly.com/news/252496560/Fujitsu-bosses... >For the first 10 years of Horizon’s existence, transaction and account data was stored on terminals in each branch before being uploaded to a central database via ISDN. Our source says this part of the system simply did not work. >“The cash account was a piece of software that sat on the counter NT box, asleep all day,” he said. “At the end of the day,…

I think a lot of "normal" people like the idea of holding corporates accountable but how would that actually work? The CEO blames one of their directors; the Director blames the supplier; the supplier blames the requirements documentation; the Business Analysts blame the culture for creating confusing and conflicting requirements. Yes, you can hold the organisation accountable but then the people who worked there bac…

> I think a lot of "normal" people like the idea of holding corporates accountable but how would that actually work?

Exactly which specific problem is "holding corporates accountable" trying to fix?

If it's that postmasters were being falsely convicted, then the way to fix that is to raise the burden of proof significantly. I hope this case has done that, and next time a court will not accept "computer says so".

With that fixed, the corporates would have to take the (falsely reported) losses; they wouldn't be able to pass it on to the postmasters like they did. Then the consequences of the problem will remain with the people responsible.

Is that sufficient?