Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sensible Software Engineering

scriptcrafty.com

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Sensible Software Engineering

#71

Post by loup-vaillant »

p1necone wrote: post #53
neo2006 wrote: post #49

Earlier quoted context omitted.

One of the mistake I see a lot around this is when people start to write intelligent mocks. Usually this is where test bugs reside.

The best is when someone writes an entirely new implementation of something just for tests, and only calls that from their tests instead of the actual product code! Then you just get blank stares when you try to explain why what they've just done is completely pointless.

The way they do it may be pointless, but the intent is not. The crux is making sure your two implementation are independent. There are two ways to achieve this. The first is clean room. Just have independent persons or teams implement the same thing. With different tools and programming languages if you can. The second is use an entirely different approach for both. The test code for instance can be constructed around the tests, and for instance rely on the fact that everything happens at the same place at the same time (when you run the test).

---

I'm currently writing on a secure handshakes generator. https://github.com/LoupVaillant/Monokex It generates specifications and source code from a Noise pattern. A bit like Noise Explorer, only with less features. Previous versions of the code were hand written, and the generated code is doing its best to look like it was hand written. Here's the result: https://github.com/LoupVaillant/Monocypher-Handshake/blob/ma...

Now how do I ensure the code match the specs? (The correctness of the specs itself is currently checked by hand.) I could write another implementation, but I'm only me, and I can't go erase my own memory to make a clean room implementation and compare the two.

Instead, I took the specs, and wrote code that takes all the inputs, and spits out all the intermediate buffers and outputs, paying no heed to stuff like order of execution, or who does what. I only concentrated on generating the test vectors: https://github.com/LoupVaillant/Monocypher-Handshake/blob/ma...

The structure of the vector generating code and the actual production code are very different. This is how I decorrelate mistakes, and make sure that if the two "implementations" agree, I'm very likely to have something that works.

---

Of course, this is all a lot of effort, so I wouldn't do this for non-critical code.

Re: Sensible Software Engineering

#72

Post by Firadeoclus »

wildmanx wrote: post #56
gilbetron wrote: post #36

Earlier quoted context omitted.

> Instead, it reduces that number to c' * N with c' That is unproven conjecture. It feels right, but just because it is written as a formula doesn't automatically mean it is correct.

Maybe we have a disconnect in our definitions of "testing". How do you define the meaning of it, and could you illustrate how it could, on average over a code base, increase the number of bugs in the product code under test?

Assuming the same time constraints, time spent writing tests competes with time spent writing product code. Ideally, tests save more time than would otherwise have been spent debugging and/or save time and money by catching bugs early.

In bad cases you get no return on the time spent writing tests and simply end up with less product code (similar bugs/LOC, so fewer bugs and reduced features). In terrible cases you end up with rushed product code and more bugs/LOC.

Re: Sensible Software Engineering

#73

Post by kolme »

danmaz74 wrote: post #66
WrtCdEvrydy wrote: post #39

Earlier quoted context omitted.

Most NodeJS with ReactJS frontends. Some of them had already started doing React for frontend and just need the backend work.

React for a js front end makes a lot of sense. But, for the back end, what advantages did they expect by moving to node - besides having the same language in the back end?

Code reuse.

For example you can use the same React components for (pre-) rendering pages on the backend so the site feels quicker to load.

Re: Sensible Software Engineering

#74

Post by runald »

pintxo wrote: post #34
gjm11 wrote: post #28

Earlier quoted context omitted.

You don't know that the test code is free of bugs. You don't need to. Case 0: No bugs in the test code. All is well. Case 1: Bug in the test code that causes some bugs in the real code not to get caught. That's bad, but you're no worse off than if you didn't have the test at all. Case 2: Bug in the test code that causes correct real code to look buggy. Result: the test fails, you look for problems, most likely you fi…

I see 5 cases: Code | Test | Result -------------------------------------------------- fine | fine | a) We have a regression test, yay! fine | buggy | b) Someone breaks the code to make the test work. Oops. c) Someone fixes the test, we have a regression test, yay! buggy | fine | d) We'll fix the code, and we have a regression test, yay! buggy | buggy | e) Bug remains in code. Oops. If the probability of introducing…

I must have missed some steps, but why did you assign p = 0.01, and how did you arrive that a) get to be (1-p) * (1-p)? And why can't, say, a) be p * p?

Re: Sensible Software Engineering

#75

Post by pintxo »

runald wrote: post #74
pintxo wrote: post #34

Earlier quoted context omitted.

I see 5 cases: Code | Test | Result -------------------------------------------------- fine | fine | a) We have a regression test, yay! fine | buggy | b) Someone breaks the code to make the test work. Oops. c) Someone fixes the test, we have a regression test, yay! buggy | fine | d) We'll fix the code, and we have a regression test, yay! buggy | buggy | e) Bug remains in code. Oops. If the probability of introducing…

I must have missed some steps, but why did you assign p = 0.01, and how did you arrive that a) get to be (1-p) * (1-p)? And why can't, say, a) be p * p?

p is the probability of introducing in error into code.

To get some actual numbers, I choose an error rate of 1 in 100 lines of code (0.01). This is totally subjective and probably larger then in reality, but it does not hurt to be pessimistic here.

a) => the production code, nor the test code contains any bug. So if the probability for an error is `p` (.01), the probability for error free code `1 - p` (.99). If both events are independent, we can multiply them to get the probability of both happening at once `(1 - p) * (1 - p)`.

Re: Sensible Software Engineering

#76

Post by danmaz74 »

kolme wrote: post #73
danmaz74 wrote: post #66

Earlier quoted context omitted.

React for a js front end makes a lot of sense. But, for the back end, what advantages did they expect by moving to node - besides having the same language in the back end?

Code reuse. For example you can use the same React components for (pre-) rendering pages on the backend so the site feels quicker to load.

You can pre-render React with Rails too (and other frameworks).

Re: Sensible Software Engineering

#77

Post by sombremesa »

neo2006 wrote: post #48
icebraining wrote: post #45

Earlier quoted context omitted.

I think what they're saying is that both the code and test will contain the bug, since the bug is in the programmer's understanding rather than just in their coding. If I understood func(5, 6) should return 9, but the client actually wants 10, no amount of tests I write will reveal the error.

I don't think that any form of formal specification will solve that issue. The only way to minimize the impact of it is to work on small iterations (agile) and be close to the client. One of the weakness in agile methodologies today is that the client representation (Product owner) is not aware of the client needs or too far from the client point of view.

This is roughly what I meant - there are ways to end up with zero bugs, but writing tests is usually insufficient for achieving such a goal.
Post reply on HN