Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The presumption that computers are working correctly

emptycity.substack.com

Post reply on HN
Print view View original

41–50 of 171 posts

Re: The presumption that computers are working correctly

#41

Post by zubairq »

bmitc wrote: post #40
zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

> I’m often surprised that computer systems work at all. I also remain contunually amazed that anything works. It boggles my mind, because I have seen how software gets built. How can it possibly all work? I think my answer to this conundrum and seeming paradox is that it doesn't all work. We have trained ourselves and been trained to accept things that aren't working as something else. And like you say, people treat…

That makes sense, yes we have trained ourselves to see things that don’t work as working , like frogs in boiling water!

Re: The presumption that computers are working correctly

#42

Post by bmitc »

phire wrote: post #7

As a programmer, it's my experience that the computer itself usually works correctly. It's almost never a hardware bug. But software.... software gets things wrong all the time and the computer hardware just blindly does what the software told it to.

> As a programmer, it's my experience that the computer itself usually works correctly. It's almost never a hardware bug.

By hardware do you mean CPU?

Also, these days, "hardware" is still something that's either got a microcontroller, FPGA, or some other thing that is eother running or was designed with software. The area is quite muddy, especially if you deal with hardware systems of a collection of various computers and hardware components.

And I'm not so sure things aren't CPU or IC bugs that get bubbled up. The amount of Heisenbugs and various other transient bugs make me question that.

In my experience with instrument control systems, the bug could be anywhere from a cable to the UI or anything in between. These systems had to be developed in strict layers to help isolate bugs and also workaround them.

Re: The presumption that computers are working correctly

#43

Post by zubairq »

kazinator wrote: post #27

Every prosecutor in possession computer evidence of a crime having to prove the software correct is a nonstarter. Assuming it's right, and proving it buggy, is the right principle.

I’m quite interested to learn and understand the line of reasoning for this comment. Care to elaborate?

Re: The presumption that computers are working correctly

#45

Post by vasco »

bmitc wrote: post #40
zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

> I’m often surprised that computer systems work at all. I also remain contunually amazed that anything works. It boggles my mind, because I have seen how software gets built. How can it possibly all work? I think my answer to this conundrum and seeming paradox is that it doesn't all work. We have trained ourselves and been trained to accept things that aren't working as something else. And like you say, people treat…

Any sufficiently complex system operates constantly in failure mode

Re: The presumption that computers are working correctly

#46

Post by amelius »

m-ee wrote: post #33
murderfs wrote: post #20

Earlier quoted context omitted.

As a programmer that works on drivers, it's my experience that hardware almost never works correctly. "We'll fix it in software" might as well be the official motto of the hardware industry.

As a firmware engineer I’m horrified at the presumption that hardware is always operating correctly. Many times I’ve been told “your new firmware is broken” and the real problem is something as mundane as an improper capacitor or resistor value. Also just look at the errata sheet for any microcontroller. Hardware bugs are so prevalent we have a special term and semi standardized document to describe them.

That's funny, because this just came in:

https://apnews.com/article/iphone-15-overheating-apple-softw...

Re: The presumption that computers are working correctly

#47

Post by lozenge »

suzzer99 wrote: post #12

This is the first I've heard of this scandal. I'm curious what the major problems were with the software that caused the discrepancies. Dropped connections? Wikipedia didn't go into much detail with that. Ok I found this: https://www.benthamsgaze.org/2021/07/15/what-went-wrong-with...

I think the issue wasn't the bug but the lack of care? Pretty much any large system develops bugs during the development but they get noticed and fixed. These bugs got noticed and not fixed.

Re: The presumption that computers are working correctly

#48

Post by gwd »

murderfs wrote: post #20
phire wrote: post #7

As a programmer, it's my experience that the computer itself usually works correctly. It's almost never a hardware bug. But software.... software gets things wrong all the time and the computer hardware just blindly does what the software told it to.

As a programmer that works on drivers, it's my experience that hardware almost never works correctly. "We'll fix it in software" might as well be the official motto of the hardware industry.

But thinking about a court case, what should be the presumption? Should it be up to the prosecution to prove that the RAID controller / network card didn't corrupt the data or that the GPU did the math correctly? Or should it be up to the defense to prove that it didn't? And how would each side demonstrate their case to a reasonable level of certainty?

As others in this thread have said, for bits of the system that have clearly defined parameters and get extensive testing (by like, millions of people), I think you can begin with the assumption that those bits are correct: you can assume the CPU is adding numbers correctly, that the compiler compiled the code correctly, that the spreadsheet executed the formula it was given correctly, and it's up to the defense to prove otherwise. Bugs in the CPU / compiler / spreadsheet software would get caught pretty quickly.

But for bits where the right answer is not clear, and / or doesn't get extensive testing -- the formulas inside the spreadsheet, custom-written software -- the assumption should be that it does not work, and it should be up to the prosecution to demonstrate that the software is working correctly.

Re: The presumption that computers are working correctly

#49

Post by reuben364 »

kazinator wrote: post #27

Every prosecutor in possession computer evidence of a crime having to prove the software correct is a nonstarter. Assuming it's right, and proving it buggy, is the right principle.

Taking this to absurdity you can create a new obfuscated program for each case that presumes guilt based on whatever evidence is could be in principle be decided with a computer, maximizing the burden on the defendants.

Maybe another principle would be that a program is assumed to function as proven in the past. The further a given program is shown prior to be correct, the more the burden of proof shifts to the defendant to show otherwise.

Re: The presumption that computers are working correctly

#50

Post by hgomersall »

bmitc wrote: post #40
zubairq wrote: post #26

After a lifetime of fixing bugs I’m often surprised that computer systems work at all. I had something similar to the failed post office system affect me personally, and everyone just rejects that the computer system could be at fault, even when presented with evidence, as it too too much work for them to try to understand.

> I’m often surprised that computer systems work at all. I also remain contunually amazed that anything works. It boggles my mind, because I have seen how software gets built. How can it possibly all work? I think my answer to this conundrum and seeming paradox is that it doesn't all work. We have trained ourselves and been trained to accept things that aren't working as something else. And like you say, people treat…

With medical devices you have to act on the assumption that the software will fail and that mitigations should be in place to deal with the consequences of failure.
Post reply on HN