91–100 of 269 posts
How do you all feel about the need to rewrite a unit test when code gets refactored or business logic changes, isn’t that like a huge pita?
I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling! I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but…
You could have watch the program and observed the failure, why need to write a test to be “surprised” it failed
Updated the original to clarify. Hope that helps!
> Now, what happens when MyThread::singlepassThreadWork() uses a member variable of MyThread like foobar and we delete the MyThread object while the thread is still running? The destruction sequence is such that MyThread is deleted first and after that, the destructor of its parent object Thread runs and the thread is joined. Thus, there is a race condition: We risk accessing the vector foobar in singlepassThreadWork() after it was already deleted. We can fix the user code by explicitly stopping the thread in its destructor
What does it mean when they say 'the destructor of its *parent* object Thread runs'? I've always thought that when you inherit from one class to another and then instantiate an object of said class, they're just one object, so what do they mean when they make the distinction between 'parent' and 'child' object? When you have inheritance of say two classes, those would be two distinct objects instantiated in memory? Is there something I'm missing?
How do you all feel about the need to rewrite a unit test when code gets refactored or business logic changes, isn’t that like a huge pita?
Not everything needs this level of rigor but there are plenty of cases where the tests are very cheap to write and reason about (for many pure functions) or are worth the cost as they validate critical behavior. Unit tests also add some design pressure to keep more logic pure/side-effect free; sure, it may take a bit more work to factor your code accordingly to keep i/o interactions separated to the shell of the application but I find this to be a useful pressure.
I've found that if I'm encountering pain when writing unit tests, then the pain is due to one of the following things:
1. The code is growing too complex and I need to decompose the logic or refactor the tests
2. The code has grown too many unintentional side effects and I need to move those side effects to discrete components
3. The code under test has fundamental side effects and those side effects require testing, thus the unit tests need to be converted to an integration test
4. The code under test is sufficiently complex that it demands full system/acceptance testing
There are some cases where refactoring the tests is generally too painful and I'll throw away all the tests entirely, maybe sprinkle in a few tests for logic that seems critical, and move on. Tests can accumulate technical debt, but in contrast to implementing code it's pretty cheap to cut your losses on tests and wipe them out.
I see a lot of people conflating unit testing with the idea that all code must have tests, and there's a ton of code that's phenomenally painful to test and can be easily checked by the developer. Tests should be a supporting tool an an augment to the developer practices; it's better to have some tests that work well and throw out the ones that are miserable to write rather than require 95% test coverage, drown in testing, and throw out all tests entirely.
It would be nice if unit tests were more like interlocking evidence of system correctness, but right now we just have integration tests with poorer coverage for that.
Earlier quoted context omitted.
Well-written breaking tests represent something changing in a code base. You can be intentional about breaking a test, but then at least you can be very explicit about what you are changing. Have seen all to many times I've broken a unit test in a code base that I did not intend to break, just to have an aha moment that I would have introduced a bug had that test not been present. Unit tests are a trade off between d…
Especially true if you get emergent side-effects from non-obvious shared state dependencies in large projects. Nightmares... =)
And pragmatically - this always happens at some point. Something something about deadlines and need to get this out yesterday.
Earlier quoted context omitted.
I think he is not implying a hard line legal standard but as connections and size increase different properties start to emerge humans start to differentiate things based on that, but there is a gradient so we can find examples that are hard to classify.
What differentiates a city from a village is legal status, not size. If size means population, there are cities with 400 inhabitants, villages with 30,000 inhabitants, and vice versa. It is not clear how this pertains to tests. When unit test was coined, it referred to a test that is isolated from other tests. Integration tests are also isolated from other tests. There is no difference. Again, the post facto attempts…
Fine. And legal status depends on location. There are many localities.
Earlier quoted context omitted.
> they prove nothing If they fail, they prove there's a bug (in either the test or the code.) This is like literally any other kind of test.
A bug in test code is not a real bug. It’s just a test that’s not giving you useful information. Lots of tests don’t give you useful information. Some that fail and some that pass. It’s easy to write a test that doesn’t provide useful information across time. Harder to write a test that does.
> It’s easy to write a test that doesn’t provide useful information across time.
I firmly believe this is one of those times.
(Currently my only issue with tests in the product I work on is that they take too long to run. Can't have it all.)
Earlier quoted context omitted.
From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…
My "unit tests" do hit the database and file system, and I have found and fixed many many problems during testing by doing so. I have found many other problems with those calls in production when I didn't do so. Yes, they make testing a lot slower. Our main app takes around 40 minutes to build which isn't good. I'd like it to be faster. But writing a bunch of separate integration tests to cover those functions would…
Tests that do these things aren’t bad. Often they are worth writing, and you generally will write them in unit test harnesses.
You hinted at the value of separating unit tests and integration tests with your observation about 40-minute unit test runs being way too slow. The process friction it creates means people will check in “obviously correct” changes without running the tests first.
Feathers continues:
However, it is important to be able to separate them from true unit tests so that you can keep a set of tests that you can run fast whenever you make changes.
You want your unit tests to be an easy habit for a quick sanity check. For the situation you described, I’d suggest moving the integration tests to a separate suite that run at least once a day. Ripping that coverage out of your CI may make you uncomfortable. That’s solid engineering intuition. Let your healthy respect for the likelihood of errors creeping in drive you to add at least one fast (less than one-tenth of a second to run is the rule of thumb from Feathers, p. 13) test in the general area of the slower integration tests.
The first one may be challenging to write. From here forward, it will never be easier than today. Putting it off is how your team got to the situation now of having to wait 40 minutes for the green bar. One test is better than no tests. Your first case with the fixture and mocks you create will make adding more fast unit tests easier down the road.
Yes, just as it’s possible to make mistakes in production code, it’s certainly possible to make mistakes in test code. Unit tests are sometimes brittle and over-constrain. Refactoring them is fair game too and far better than throwing them away.
I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…