Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What I wish I knew when I became CTO

medium.com

Post reply on HN
Print view View original

201–210 of 258 posts

Re: What I wish I knew when I became CTO

#201

Post by deckard1 »

arekkas wrote: post #176
forgotpw1123 wrote: post #146

Earlier quoted context omitted.

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Let's break this down. > I'm convinced that unit tests don't usually find bugs. They don't, they test whether or not the API contract the developer had in mind is still valid or not. > IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. You don't write test to find bugs (in 98% of cases), but you can write tests for bu…

> Developers must have the capability of quickly testing the system without manual work

Running unit tests is hardly quick. Especially if you have to compile them. End-to-end are even worse, in this regard.

> They don't, they test whether or not the API contract the developer had in mind is still valid or not.

If you're always breaking the API, then that's a sign that the API is too complex and poorly designed. The API should be the closest thing you have to being set in stone. Linus Torvalds has many rants on breaking the Linux kernel's API (which, also, has no real unit tests).

It's also really easy to tell if you're breaking the API. Are you touching that API code path at this time? Then yes, you're probably breaking the API. Unless there was a preexisting bug that you are fixing (in which case, the unit test failed) then you are, by definition, breaking the API, assuming your API truly is doing one logical, self-contained thing at a time as any good API should.

edit: As an aside, I'd like to point out that POSIX C/C11/jQuery/etc. are littered with deprecated API calls, such as sprintf(). This is almost always the correct thing to do. Deprecate broken interfaces and create new interfaces that fix the issues. Attempting to fix broken APIs by introducing optional "modes" or parameters to an interface, or altering the response is certain to cause bugs in the consumer of the interface.

> Don't abandon good development practices

Unit tests are a tool. There are cases where they make sense, where they are trivial to implement and benefit you greatly at the same time. Then there are cases where implementing a unit test will take an entire day with marginal benefit and the code will be entirely rewritten next year anyway (literally all web development everywhere). It doesn't make sense to spend man-months and man-years writing and maintaining unit tests when the app will get tossed out and rewritten in LatestFad Framework almost as soon as you write the test.

Re: What I wish I knew when I became CTO

#202

Post by arkh »

Rapzid wrote: post #73
mkarazin wrote: post #13

> I’ve found it a real struggle to get our team to adopt writing tests. I find this hard to believe. Do others CTOs / team leads find this to be the case? I've been a CTO of two small startups with 3-7 developers. We've had resistance to tests at some points (myself included). We've solved it fairly simply. All pull requests (PRs) require tests. PRs are rejected immediately without tests. If a PR doesn't have tests a…

How many tests does a PR need? One? Five? When would you not write tests for something(that something not necessarily being a PR, but maybe a unit or feature)? Tests, like any process, should be serving you and your goals.. You shouldn't be serving your processes or testing practices. This sort of un-nuanced thinking isn't indicative of a high performing startup or CTO IMHO. Perhaps your policies are not directly ind…

> How many tests does a PR need?

Enough test for each of your spec. Adding a new functionality to your product? Your test should cover and the cases you put in your specs. Correcting a bug? You test should trigger it with the old code.

You can have 100% code coverage with unit testing, it will do jack-shit for your users when they enter a negative number and there was no code meant to manage this case so it never was tested.

Re: What I wish I knew when I became CTO

#203

Post by apocalyptic0n3 »

muttech wrote: post #41
apocalyptic0n3 wrote: post #23

Earlier quoted context omitted.

I'm not a CTO but I do lead the dev team at our agency (was previously 16 devs, but we've slimmed down to 7 currently). I want to preface this by saying that at an agency, your biggest enemy is always time; sales teams have to sell projects for the absolute minimum in order to get a contract, so you can't waste time on non-essentials for most projects. That said, the biggest resistance I have found is "this feature i…

In my previous agency life, this was something that I experienced as well. A short lived product that was due in less time than any sane dev would estimate. We all knew that we "should" write tests, but there just wasn't time. And in 6 weeks the project would be relegated to living in source control because the campaign was over. It made hiring devs fun. Trying to explain to people why it was that way, and their insi…

> A short lived product that was due in less time than any sane dev would estimate. > And in 6 weeks the project would be relegated to living in source control because the campaign was over.

That is exactly it for 90% of agency projects. Underquoted to get the deal, a rapid development cycle that leaves the devs feeling dead, and then once that first release is out, you have maybe 1 or 2 small updates and the project is never touched again, or at least not for a year or two.

There is no world where it makes sense to write tests for these projects.

Re: What I wish I knew when I became CTO

#204

Post by virtualized »

dangero wrote: post #143
snowwrestler wrote: post #138

Earlier quoted context omitted.

I think the theory is that increasing quality will save time in the long run via fewer regressions and bug reports. Making that math work, though, seems to depend on the idea of some sort of future crisis state, where normal development is slowed way down. (You'd need to avoid a big slow down in the future, in order to balance out the continuous extra time given to testing.) Does such a crisis lurk in the future of e…

Exactly. It’s likely for some projects every line of code you write will in the end be a complete waste. TDD is only worth the trouble assuming any of the project succeeds. If not it may just be more wasted effort. For this reason I think a case can be made to skip TDD on MVP traction tests in some cases. A friend of mine just had his startup acquired, so his startup was an above average success and he told me 80% of…

The problem is you don't know which 80% will get scrapped.

Re: What I wish I knew when I became CTO

#205

Post by apocalyptic0n3 »

nine_k wrote: post #121
bowlich wrote: post #77

Earlier quoted context omitted.

I remember my agency days. Now days I work on a single long-running legacy project where tests make sense. Back then, I read a lot about how testing was the "right thing to do." But I also realized that most of the time (a) the client wasn't going to be willing to pay for the tests and (b) odds are that once we launch the product, that will be last time I ever look at it. Maintenance will occur in five years when sal…

It's a very interesting setup: all prod code is "throwaway". I wonder if each such project is built completely from scratch. If not, the reusable parts can be improved over time, and covered by tests.

> I wonder if each such project is built completely from scratch.

For us, it's a mixed bag. We have a CMS we use for most projects that we did not develop, but we have developed our own packages/blocks for it that are included in every project that bootstrap and whitelabel the hell out of the CMS to provide the functionality we need in every project. From a data standpoint, one of our packages replaces several dozen classes and hundreds, if not thousands, of lines of custom code in every project.

When it comes to more custom projects, specifically ones that never see public use (like a custom CRM, admin dashboard, CRUD-based system, API backend, etc.), we build using the Laravel framework which bootstraps away all of the authentication, permissions, routing, middleware, etc. and gives us a very good blank slate to work with. For these, everything is mostly from scratch, minus what we can use third-party packages for (such as the awesome Bouncer ACL). We have a front-end library that I wrote to abstract away common tasks into single line instantiations, but it's our experience that these projects are being built on a blank slate for a reason. These are the projects that may actually see tests written for as well, although not all will.

Re: What I wish I knew when I became CTO

#206

Post by apocalyptic0n3 »

yread wrote: post #162
apocalyptic0n3 wrote: post #23

Earlier quoted context omitted.

I'm not a CTO but I do lead the dev team at our agency (was previously 16 devs, but we've slimmed down to 7 currently). I want to preface this by saying that at an agency, your biggest enemy is always time; sales teams have to sell projects for the absolute minimum in order to get a contract, so you can't waste time on non-essentials for most projects. That said, the biggest resistance I have found is "this feature i…

I always find code coverage such a useless metric: if you have two independent ifs next to each other and one test goes in one if and another test in the other you have 100% coverage. Congratulations. But you've never tested what happens when you go in both

I agree that it is a useless statistic, especially when comparing unit vs integration vs functional vs smoke testing. There are different types of tests and just because you are reaching 90% of your code does not mean you are thoroughly testing it.

The only reason I brought it up was to show that we don't skip test writing entirely and the projects where we do write them, it isn't like we just wrote a test to check that "Project Name" is returned on the homepage and called it a day.

Re: What I wish I knew when I became CTO

#207

Post by apocalyptic0n3 »

mikekchar wrote: post #85
apocalyptic0n3 wrote: post #23

Earlier quoted context omitted.

I'm not a CTO but I do lead the dev team at our agency (was previously 16 devs, but we've slimmed down to 7 currently). I want to preface this by saying that at an agency, your biggest enemy is always time; sales teams have to sell projects for the absolute minimum in order to get a contract, so you can't waste time on non-essentials for most projects. That said, the biggest resistance I have found is "this feature i…

I find that tests pay off pretty quickly in terms of productivity -- somewhere around a week. There are a couple of caveats, though. First, you have to have a team that's already good at TDD (and not just test-after-the-fact). What I mean by TDD is hard to describe succinctly and especially since I said it's not test-after-the-fact, it's easy to think that I mean test first. I don't. To me TDD is a way of structuring…

> I find that tests pay off pretty quickly in terms of productivity -- somewhere around a week.

I think you overestimate the agency project life cycle. Most of our projects are built and ready for client review in 2-3 weeks total. Once the client makes a few days worth of changes, the project is shipped and we likely do not look at it again for another year or three.

That said, there are always long-running projects and those are the ones you try to include tests in.

Re: What I wish I knew when I became CTO

#208

Post by akvadrako »

forgotpw1123 wrote: post #146
latch wrote: post #65

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniq…

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Unit tests are there to make the code less fragile, so that it can be modified with confidence. But if you need tests to make your code robust, it's likely a mess underneath; probably better to spend the time refactoring.

Personally, I say write tests when it makes development quicker or serves as a good example / spec.

Re: What I wish I knew when I became CTO

#209

Post by noxecanexx »

neeleshs wrote: post #182
arekkas wrote: post #176

Earlier quoted context omitted.

Let's break this down. > I'm convinced that unit tests don't usually find bugs. They don't, they test whether or not the API contract the developer had in mind is still valid or not. > IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. You don't write test to find bugs (in 98% of cases), but you can write tests for bu…

Indeed. Tests allow new members of the team to confidently make changes. I've seen codebases that had near zero tests and also a total mess, with one change somewhere breaking a hundred things 30 levels down the stack. We'd find the issue only in production, along with an enraged customer. Tests are not a replacement for good developers, they are just a tool for contract validation and a regression safety net.

Actually no. For contract validation use type system or something like clojure.spec . I only writes tests to prevent regression or to prove a feature/PR works.

Re: What I wish I knew when I became CTO

#210

Post by akvadrako »

crdoconnor wrote: post #171
forgotpw1123 wrote: post #146

Earlier quoted context omitted.

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

I think unit tests will die one day, and that day is probably not too far away. These days I follow three "good practice" rules, all of which are violated when you follow common unit testing practise: * Only put tests on the edge of a project. If you feel like you need lower level test than that then either a) you don't or b) architecturally, you need to break that code off into a different project. * Test as realist…

> Only put tests on the edge of a project.

I mostly agree with your point, but I think this is too much. Projects should be made up of decoupled modules (units ?) with well-defined interfaces. These should be stable and tested, and mostly without mocking required.

The larger your project the more important this is.

Post reply on HN