Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making Software Reliable: The Importance of Testability

codereliant.io

Post reply on HN
Print view View original

11–20 of 25 posts

Re: Making Software Reliable: The Importance of Testability

#11

Post by mgbmgb »

gjsman-1000 wrote: post #2

I'm actually rather negative on automated testing. Every project, that has gotten rid of the Q/A team to rely on automated tests, hasn't exactly gone well. Windows being the obvious example, but there are others. I'm not seeing the quality improvement that should be there. And even when there are tests, elementary mistakes (like the time Windows 10 would delete your Documents folder) slip through the pipeline and scr…

I wouldn't dismiss the value of automated testing altogether, I think it really depends on the domain. I haven't seen a dedicated QA team in any of my recent companies and all the projects do just fine.

With that said, if I were to develop something high-profile I'd use a combination of both (and usually funding is not an issue for this types of projects)

Re: Making Software Reliable: The Importance of Testability

#12

Post by mgbmgb »

sethammons wrote: post #7

can't access the article, but I'm big on testability. We used to have to watch each change like a hawk via monitoring and manual checks. We automated with SLO monitoring for prod and heavy use of gated builds protected behind integration and acceptance tests. My favorite system designed and used so far: PR -> gated integration build with docker-compose -> merge master -> async master/main re-run same test to prevent…

This is the way to go. With the recent push towards "more nimble teams" and high output / high velocity there is no place for QA teams to take days to review the releases, so you'd have to bake testability in.

Recent example how fairly small team in Meta built the Threads app https://engineering.fb.com/2023/09/07/culture/threads-inside...

Re: Making Software Reliable: The Importance of Testability

#13

Post by gjsman-1000 »

gjsman-1000 wrote: post #9
ChrisMarshallNY wrote: post #6

Earlier quoted context omitted.

You can do that in Swift (and, I suspect, lots of languages). Swift has a fairly decent assertion/precondition facility, as well as reflection[0]. They also have a decent test framework[1] (and I know that some folks have extended it, to do some cool stuff). Some of these add significant overhead, so they aren't practical for shipping runtime, but they can be quite useful for debug-mode validation. Assertions are a v…

> Of course, all the tools in the world, are worthless, if we don't use them. True... I wonder if there would be any way, to simplify the syntax, and require basic assertions to be on every function. There might be a super easy cop-out like `any`, but at least by being forced to type it, you become aware of what it means and that it exists. Almost like: `public any int addXandY (any int x, any int y) {` I also wonder…

I just had another thought. What if you could have a bank of assertions? Like this pseudocode:

```

assertion acceptableBlastNumber (int x) { x 5; }

assertion acceptableBlastRadius (int x) { x > 500; x 750 && x assertion acceptableBlastAddedNumber (int x) { x 505; }

public acceptableBlastAddedNumber int addBlastNumbers (acceptableBlastNumber int x, acceptableBlastRadius int y) { return x + y; }

addBlastNumbers (10, 720) => 730

addBlastNumbers (26, 750) => Exception

```

Though I suppose that this is getting really close to just... classes. It would just be a little more... inline? Less complicated because it would never hold state? Though I suppose, this would also mean your class can just focus on being an objec, and not on having all the definitions for the things inside it, because you can have an rather than just .

Re: Making Software Reliable: The Importance of Testability

#14

Post by jjice »

Using interfaces (whether literal interfaces in your language or the general concept) to isolate operations that rely on dependencies is so critical IMO. It makes testing so much easier, takes little additional upfront effort, and generally makes nicer to read code (subjectively).

For example, if I have my `MailSender` interface with an `SMTPMailSender` implementation but then I want to switch over to using the AWS SDK for interacting with SES for auth purposes, I can just implement my interface as always and create my new `SESMailSender` and plop it in place and all my code just works. This isn't just for testability, but also general modularity.

It takes so little additional upfront effort as well. Even just declaring an interface helps you isolate what functionality should actually be publicly exposed. I really don't think this creates any serious development drag. On top, it makes your tests so much easier to write, and tests are usually a good bang for your buck for the major cases.

If you truly only need a single implementation and not even the ability to have one for testing, then that's fine! Keeping in mind that adapting this to an interface in the future should be simple and do that when it comes up.

Re: Making Software Reliable: The Importance of Testability

#15

Post by RcouF1uZ4gsC »

One concern I have with designing for testability (which often means, designing for ease of unit testing) is that everything becomes a abstract interface.

Whereas before you might have

  void Foo(){
    Bar();

  }

  void Bar(){
    Baz();
  }

  void Baz(){

  }
Where Bar and Baz only really have a single implementation

This is fairly easy to grok and debug.

You end up with

   void Foo(BarInterface* bar, BazInterface* baz){

   bar->Bar(baz);

   }
Now you have a lot more moving parts. If you have a problem with Foo, now you need to try to find out who all implements the interfaces and which implementation is being passed to.

Of course, after a while people get tired of passing in these interfaces and come up with some kind of "automated" dependency injection framework, which then triples the complexity.

Re: Making Software Reliable: The Importance of Testability

#16

Post by jandrewrogers »

gjsman-1000 wrote: post #5
diekhans wrote: post #4

Earlier quoted context omitted.

My view is that automated testing is not a substitute for QA, but an additional tool. It lets QA focus on harder to automated tasks. For unit tests, a developer is going to try something out anyway, so capturing it in a unit test for the future should be just a little extra work. Also, writing a unit test means the developer has minimally used what they are writing. Higher-level (system) testing, especially with GUIs…

There is one programming language that fascinated me (maybe it was Ada) where it tried to have some basic tests inline with the code, by defining basic guidelines for legitimate results of the function. For example, you could make a function called `addLaunchThrusterAndBlastRadius` (I know it make no sense, but bear with me), and then right alongside declaring it was an integer, you could put a limit saying that all…

Modern C++ supports this pretty extensively via the type system. You can define/construct integer types with almost arbitrary constraints and properties that otherwise look like normal integers, for example. The template / generics / metaprogramming / type inference facilities in C++ make it trivial. Some categories of unsafe type interactions can be detected at compile-time with minimal effort, it isn't just runtime asserts.

This is common in C++ for reliable systems. You infrequently see a naked 'int' or similar (usually at OS interfaces), almost all of the primitive types are constrained to the context. It is a very useful type of safety. You can go pretty far with a surprisingly small library of type templates if the constraint specification parameters are flexible.

(This is also a good exercise to learn elementary C++ template metaprogramming. A decent constrained integer implementation doesn't require understanding deep arcana, unlike some other template metaprogramming wizardry.)

Re: Making Software Reliable: The Importance of Testability

#17

Post by gbacon »

jjice wrote: post #14

Using interfaces (whether literal interfaces in your language or the general concept) to isolate operations that rely on dependencies is so critical IMO. It makes testing so much easier, takes little additional upfront effort, and generally makes nicer to read code (subjectively). For example, if I have my `MailSender` interface with an `SMTPMailSender` implementation but then I want to switch over to using the AWS S…

Combine that with the abstract test pattern, and teaching your harness how to instantiate an instance of a new concrete implementation is often all that’s necessary.

Re: Making Software Reliable: The Importance of Testability

#18

Post by fatnoah »

diekhans wrote: post #4
gjsman-1000 wrote: post #2

I'm actually rather negative on automated testing. Every project, that has gotten rid of the Q/A team to rely on automated tests, hasn't exactly gone well. Windows being the obvious example, but there are others. I'm not seeing the quality improvement that should be there. And even when there are tests, elementary mistakes (like the time Windows 10 would delete your Documents folder) slip through the pipeline and scr…

My view is that automated testing is not a substitute for QA, but an additional tool. It lets QA focus on harder to automated tasks. For unit tests, a developer is going to try something out anyway, so capturing it in a unit test for the future should be just a little extra work. Also, writing a unit test means the developer has minimally used what they are writing. Higher-level (system) testing, especially with GUIs…

> My view is that automated testing is not a substitute for QA, but an additional tool. It lets QA focus on harder to automated tasks.

This is my take as well. Automation is great for a lot of the repetitive work, but humans are better at creatively breaking stuff, handling UX testing, and improving and enhancing the automation itself.

Re: Making Software Reliable: The Importance of Testability

#19

Post by mrkeen »

RcouF1uZ4gsC wrote: post #15

One concern I have with designing for testability (which often means, designing for ease of unit testing) is that everything becomes a abstract interface. Whereas before you might have void Foo(){ Bar(); } void Bar(){ Baz(); } void Baz(){ } Where Bar and Baz only really have a single implementation This is fairly easy to grok and debug. You end up with void Foo(BarInterface* bar, BazInterface* baz){ bar->Bar(baz); }…

> If you have a problem with Foo

If you have a problem with Foo, you instantiate a Foo inside FooTest, and then feed it problematic input.

You can't instantiate a Foo inside FooTest unless you can instantiate a Foo and a Bar. Foo could be a clock whose time-of-day you can't control, and Bar could be a database that you can't instantiate.

Post reply on HN