271–280 of 329 posts
I used to write tests for all pure functions because they were the easiest tests to set up. They are also the easiest to debug so the test didn't really help unless you're checking for type signatures.
I think that implementation tests are important but I found that I suck at figuring our how to set up a test before the actual implementation. So I do them after the fact and judiciously.
No. Tests are like any other code. They incur technical debt and bugs at the same rate as other code. They also introduce friction to the development process. As your test suite grows, your dev process often begins to slow down unless you apply additional work to grease the wheels, which is yet another often unmeasured cost of testing in this fashion. So, in short, I view tests as a super useful, but over-applied too…
At least 50% of my last job was writing tests, and the snail's pace of their dev process was the main reason I left.
Nope. I pretty much always find it to be counterproductive. Most of programming happens in the exploration phase. That's the real problem solving. You're just trying things and seeing if some api gives you what you want or works as you might expect. You have no idea which functions to call or what classes to use, etc. If you write the tests before you do the exploration, you're saying you know what you're going to fi…
> You're just trying things and seeing if some api gives you what you want or works as you might expect. I don't do most of my programming this way, because mostly I'm writing new things, not gluing together existing APIs with a tiny amount of simple glue code. But when I do need to characterize existing APIs, I find that unit tests are a really helpful way to do it — especially in languages without REPLs, but even i…
Earlier quoted context omitted.
Do you consider coverage an effective metric? I've got some code that has a test suite which is effectively a bunch of low level driver checks, plus a bunch of common example snippets and checks for eg empty inputs etc. Coverage gives an idea of how many lines of code have been run, but obviously no guarantees of correctness for those specific lines (eg you can't detect a double negative). It's worked well for me so…
I definitely think coverage is a worthy metric to track. It can provide meaningful information about the "doneness" of your tests. It shouldn't drive testing though, and especially, you shouldn't write your tests specifically "to get coverage". Yes, lots of people do this in environments where "getting 100% coverage" is mandatory. That said, I've found issues specifically after targetting blocks for testing, which we…
This is the single most important part of code coverage IMO: we don't care about what the tests cover, we care about what the humans never considered.
For this reason I'm a proponent of 100% coverage with a major caveat: Any code you explicitly decided not to test gets marked "no cover", so it doesn't count for or against the coverage score. This way branches that were accidentally missed really stand out, and we're not bogged down by having to test 100% of the code.
Earlier quoted context omitted.
Maybe you can help me understand this. Since you don't write as many tests, that means you're not actually testing all your code branches because tests incur technical debt after all. So does this mean you test every single branch manually? just don't bother with it at all? Do you just have a few integration tests and they break and you spend a good chunk of time figuring out which logical branch broke? What happens…
Yeah, as soon as code has more than two "real" branches, I don't trust myself to manually test them all. One of them will be broken quickly if I keep hacking in a particular branch. (This is, secretly, also an argument for writing code in sufficient generality to avoid this phenomenon in the first place.) I also never trust a test that passes the first time I run it. I am both terrible at writing correct code, and co…
No. Tests are like any other code. They incur technical debt and bugs at the same rate as other code. They also introduce friction to the development process. As your test suite grows, your dev process often begins to slow down unless you apply additional work to grease the wheels, which is yet another often unmeasured cost of testing in this fashion. So, in short, I view tests as a super useful, but over-applied too…
Maybe you can help me understand this. Since you don't write as many tests, that means you're not actually testing all your code branches because tests incur technical debt after all. So does this mean you test every single branch manually? just don't bother with it at all? Do you just have a few integration tests and they break and you spend a good chunk of time figuring out which logical branch broke? What happens…
Thanks god this is handled out of the box by any compiled language. I can't imagine myself writing tests to catch non-logical bugs.
No. Tests are like any other code. They incur technical debt and bugs at the same rate as other code. They also introduce friction to the development process. As your test suite grows, your dev process often begins to slow down unless you apply additional work to grease the wheels, which is yet another often unmeasured cost of testing in this fashion. So, in short, I view tests as a super useful, but over-applied too…
Thankfully we don't have to. Hillel Wayne links to a few of the studies that have been done on TDD [0]. It doesn't have a conclusive effect on error rates in software. While I do write tests before I write code, more so in a dynamic language without a strong, static type system; it appears that there isn't any correlation to a reduced number of bugs. But I still do it. And I think that's because that while I may prev…
To put it another way, it is not surprising that error rates are similar since, presumably, you keep coding till you produce an error, fix it and then repeat.
The question is how much error free code you wrote between the errors. If testing -- or any technique for that matter, like relaxing, jogging, discussing, planning -- reduces errors, it will manifest as more functionality per error.
Almost never. With the kind of software I mostly write these days, I'm fortunate to be able to incrementally develop my code and test it under real-world conditions or a subset thereof. So my approach is exploratory coding -- I start with minimum workable implementations, make sure they work as needed, and then add more functionality, with further testing at each step. The upside is that I don't have to write "strang…
> With the kind of software I mostly write these days, I'm fortunate to be able to incrementally develop my code and test it under real-world conditions or a subset thereof. What kind of software you write if you don't mind me asking ?and are your "real-world conditions" tests automated ? > The upside is that I don't have to write "strange" code to accommodate testing. Can you elaborate more as what you mean by "stra…
Yes, this is kid stuff. My current work is about as sophisticated as typical undergrad Computer Science projects. We can't all be doing rocket science!
I used to write automated test setups for my programs, providing streams of pre-canned messages and such. That worked out OK. I suppose it's great to have test suites to avoid regression and such, but I ended up regretting all the effort I sunk into testing. So far it's been my experience that I would sink a lot of time into creating a test suite that could exercise my programs as thoroughly as simple exposure to real-world message traffic.
I hope my attempt to be brief didn't come across as derogatory when I wrote "strange." Here's an example: I like to make a lot of my fields and methods private. It's handy that my IDE warns me when fields and methods aren't used, or when final fields aren't initialized. Obviously, for "classic" unit tests I'd have to at least expose my methods at the package level to call them from out of class. Another example: my apps rely on a fair bit of configuration data and some embarrassingly tight coupling between my classes. A JUnit-friendly program would call for a lot of mockups, as well as a lot more coding to interfaces rather than concrete classes, probably a lot more reliance on design patterns. My coding style for these projects yields a small number of compact classes but is very hostile to unit testing.
To be clear: For many other projects, your mileage may vary dramatically. I've successfully done TDD in other projects where that made a lot more sense.
Of course, there are always exceptions. if you have software that is highly complex but the outputs are very simple and easy to measure then it might actually be a good idea.