Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Programmers should never trust anyone, not even themselves

carbon-steel.github.io

Post reply on HN
Print view View original

111–120 of 163 posts

Re: Programmers should never trust anyone, not even themselves

#111

Post by bloopernova »

dartos wrote: post #64
atribecalledqst wrote: post #60

> Failing tests indicate the presence of bugs, but passing tests do not promise their absence. As somebody who primarily lives on the testing side of the house, I've definitely run into cases where the developer promises that their unit tests will make a new feature less buggy, then about 5 minutes later I either find a mistake in the test or I find a bug in something that the developer didn't think to test at all. I…

I’ve been preaching this for a while. When a codebase gets too big, and devs gets too clever with their tests, the whole test suite becomes complicated. If your test suite is approaching the complexity of the actual codebase (what with layers of mocks and fixtures that are subtly interdependent,) how could you be expected to trust a test you wrote more than the code you wrote.

I (sysadmin/devops) am writing some nodejs and the complexity of the tests is confusing to me. I'm a nodejs beginner to be sure, and I'm not experienced enough to verify what copilot gives me.

All those mocks, and other Jest code, all seem overly complicated but I don't know of anything "better".

Re: Programmers should never trust anyone, not even themselves

#112

Post by williamdclt »

sameoldtune wrote: post #93
atribecalledqst wrote: post #60

> Failing tests indicate the presence of bugs, but passing tests do not promise their absence. As somebody who primarily lives on the testing side of the house, I've definitely run into cases where the developer promises that their unit tests will make a new feature less buggy, then about 5 minutes later I either find a mistake in the test or I find a bug in something that the developer didn't think to test at all. I…

My gift to any team I work with is an integration test harness that usually has some kind of DSL for setting up state. This looks wildly different depending on the project. But my theory is that if tests are easy to write then it is easy to make more of them. So it is worth it to write some ugly code one time under the hood to make this happen. If every test requires copy pasting a bunch of sql statements and creatin…

we have _exactly_ that (down to the `withXXX` syntax), and indeed I found it great.

Downside is that test setup can be a bit slow: each `withXXX` can create more data than really necessary (eg a "withPost()" might update some "Timeline" objects, even though you really don't care about timelines for your test). Upside is that it's a lot closer to what happens in reality, regularly finding bugs as side-effect. And also you align incentives: you make your tests faster my making your application faster.

Re: Programmers should never trust anyone, not even themselves

#113

Post by agentultra »

n4r9 wrote: post #40
psychoslave wrote: post #16

> verifying code correctness is impossible Somehow, taking into account the state of our industry, yes. But this is not an absolute truth. I mean, we do have the theoretical frameworks and even tools to come with solutions that allow to proof that code is correct. It’s just that mapping this "know how" with the "how to deal with the expected flow rate feature" is very uncommon.

> tools to come with solutions that allow to proof that code is correct I may be misunderstanding, but isn't part of the problem that these tools are themselves written in code and therefore subject to bugs?

This is addressed by the de Bruijn Criterion. The essential idea is that a small number of "trusted" rules should be enough to satisfy even the largest proofs. You have to keep the number of rules small enough that they can be reviewed and understood by humans so that you can trust the proofs verified by the kernel.

Re: Programmers should never trust anyone, not even themselves

#114

Post by ImHereToVote »

Spivak wrote: post #108
fatnoah wrote: post #95

Earlier quoted context omitted.

> "Trust, but verify" Even though my entire career has been software, I was an Electrical Engineering major, so I have taken VLSI design (and even designed an 8-bit ALU). My first job was writing embedded software, and would frequently "trust, but verify" the hardware through the use of a logic analyzer. When I pulled out printouts from the analyzer to show the hardware team that the hardware had a bug, the surprised…

This is why blameless postmortems are such good thing. Going one step further having a culture where finding a bug or defect in your own code/design is rewarded makes it so people aren't afraid but excited to talk about them.

"Blame is for small children and God"

Re: Programmers should never trust anyone, not even themselves

#115

Post by agentultra »

To me, this is the argument for formal verification. I don't want to hear a hand-waving explanation that this algorithm will always complete. If the algorithm is sufficiently complex I want proof. Otherwise, why would I believe you?

Abstractions, in the mathematical sense, always hold (unless there is a flaw in the definition itself). Axioms in any sense are always going to throw a wrench in things. Thank Godel. But that shouldn't mean we cannot make progress.

Do the work, show your proof! Think hard!

Although sometimes all you need are a few unit tests.

They key is to develop the wisdom to know when unit tests aren't sufficient for the task at hand.

Re: Programmers should never trust anyone, not even themselves

#116

Post by bigstrat2003 »

kryptiskt wrote: post #48

I think "trust, but verify" (as mentioned in the article) is a much more useful motto than "never trust anyone". The latter isn't an useful attitude, if you took it seriously you would have carefully check or rewrite everything from the ground up. And then you'd either have to trust the hardware anyway or enlist in a course on VLSI design. "Trust, but verify" is much more practicable, at least if you don't feel the n…

If you have to check up on someone to make sure they aren't screwing up (or misleading you), then you don't actually trust them. Thus, "trust but verify" is not trust at all.

Re: Programmers should never trust anyone, not even themselves

#117

Post by marcosdumay »

Spivak wrote: post #110
marcosdumay wrote: post #94

Earlier quoted context omitted.

> I've also seen instances where tests are written too early, using a data structure that gets changed in development That's as clear a signal that they are testing the wrong interface as you can get. Unfortunately, developers think of tests as testing code, not interfaces. As a natural consequence, they migrate towards testing the most complex break-down of their code as they can; it increases the ratio of code cove…

There's two kinds of tests I think developers are trying to write and I think both of them have merits. Writing a test for the code is actually totally fine so long as the reason that you're doing it is because you want to be able to depend on that and you need something to scream if ever changes. I think starting with a test that the code does what the code does is actually a pretty good starting point because it's…

I get the impression you are talking about taking over legacy code.

Well, the invariants you want to test are what people want the software to do. If you create those test from the beginning, there's nothing to guess. But of course, if people write a bunch of code and throw that knowledge away, you have to recover it somehow, and it will necessarily involve a lot of guessing.

Re: Programmers should never trust anyone, not even themselves

#118

Post by vzaliva »

As someone who routinely formally verifies code, I can confirm that you should not trust yourself. I keep finding bugs and hidden assumptions in the most trivial code or even in code covered with unit tests. In my opinion, formal verification is the only way to truly trust some code.

Re: Programmers should never trust anyone, not even themselves

#119

Post by n4r9 »

agentultra wrote: post #113
n4r9 wrote: post #40

Earlier quoted context omitted.

> tools to come with solutions that allow to proof that code is correct I may be misunderstanding, but isn't part of the problem that these tools are themselves written in code and therefore subject to bugs?

This is addressed by the de Bruijn Criterion. The essential idea is that a small number of "trusted" rules should be enough to satisfy even the largest proofs. You have to keep the number of rules small enough that they can be reviewed and understood by humans so that you can trust the proofs verified by the kernel.

That probably helps to reduce the occurrence of issues, but I feel you are still ultimately relying on the correctness of proof assistants like Coq. And I am sure that bugs are occasionally found in Coq!

Re: Programmers should never trust anyone, not even themselves

#120

Post by dncornholio »

robertlagrant wrote: post #44
dncornholio wrote: post #34

Earlier quoted context omitted.

Has nothing to do with trust. I trust that you will give me the money. I also trust that you counted the right amount. But I still want to make sure you did not make a mistake.

I suppose the question is: why bother with trust? Why not just "verify"?

Well the thing is, that’s how trust already works isn’t it? Trust doesn’t require any effort. Trust means I won’t put in effort. So I just verify, because I trust.
Post reply on HN