Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What I wish I knew when I became CTO

medium.com

Post reply on HN
Print view View original

181–190 of 258 posts

Re: What I wish I knew when I became CTO

#181

Post by robinwarren »

Re: getting your team more interested in testing. This is not an easy thing to get momentum on if people aren't used to it. Yes to getting the test time down (and keeping it down)

Also, try defining (maybe in collaboration with the team) the tests you want people to write rather than leaving it up to them or (hopefully not) expecting 100% coverage. I wrote this on my thoughts a while back https://getcorrello.com/blog/2015/11/20/how-much-automated-t... We had some success with increasing testing using that and code review so others could check tests were being written. Still not total buy in to be honest but a big move in the right direction :)

One surprising thing was that after years of thinking I was encouraging my team to write tests, the main feedback on why they didn't was that the didn't have time. Making it an explicit part of the process and importantly defining what tests didn't need to be maintained forever really helped.

Re: What I wish I knew when I became CTO

#182

Post by neeleshs »

arekkas wrote: post #176
forgotpw1123 wrote: post #146

Earlier quoted context omitted.

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Let's break this down. > I'm convinced that unit tests don't usually find bugs. They don't, they test whether or not the API contract the developer had in mind is still valid or not. > IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. You don't write test to find bugs (in 98% of cases), but you can write tests for bu…

Indeed. Tests allow new members of the team to confidently make changes. I've seen codebases that had near zero tests and also a total mess, with one change somewhere breaking a hundred things 30 levels down the stack. We'd find the issue only in production, along with an enraged customer.

Tests are not a replacement for good developers, they are just a tool for contract validation and a regression safety net.

Re: What I wish I knew when I became CTO

#183

Post by afarrell »

rmccoy6435 wrote: post #135
mratzloff wrote: post #111

Earlier quoted context omitted.

This stems from an unwillingness to make it a job requirement. There are several things you as a software engineer are expected to do as a part of your job: write code, write tests, participate in code reviews, ensure successful deployment, work effectively with various groups, etc. It's really simple: state the job requirements up front in the position description and during the hiring process. Make testing part of…

I'm in 100% agreement with you up until the point of tying your test coverage and writing of tests to your employment. In my eyes that promotes a culture of writing bogus tests that provide no value other than to make more green check marks. You should be encouraged to write tests by your colleagues and be in a culture that sees the benefits, rather than forcing people to do it. I'm also unsure if sitting one develop…

If a PR has bogus tests that provide no value other than to make more green check marks, how do they pass code reviews? That indicates that your code review process is kinda broken--tests should support the code review process by indicating what edge cases the writer of a PR has thought of and then prompting the reviewer to ask what hasn't been thought of.

Re: What I wish I knew when I became CTO

#184

Post by mratzloff »

rmccoy6435 wrote: post #135
mratzloff wrote: post #111

Earlier quoted context omitted.

This stems from an unwillingness to make it a job requirement. There are several things you as a software engineer are expected to do as a part of your job: write code, write tests, participate in code reviews, ensure successful deployment, work effectively with various groups, etc. It's really simple: state the job requirements up front in the position description and during the hiring process. Make testing part of…

I'm in 100% agreement with you up until the point of tying your test coverage and writing of tests to your employment. In my eyes that promotes a culture of writing bogus tests that provide no value other than to make more green check marks. You should be encouraged to write tests by your colleagues and be in a culture that sees the benefits, rather than forcing people to do it. I'm also unsure if sitting one develop…

Bogus tests have to be caught in code review. When I talk about educating the team that's what I mean.

I've only ever had to do a test rotation once or twice, and it was like pulling the rip cord on a lawnmower. Requires effort at first and then it becomes self-sustaining over time. It establishes or affirms a culture of testing. The rotation doesn't even need to last long.

You should know which portions of the code are here to stay and which are nearing their end of life. Naturally, you want to spend your time where it will have maximum payoff.

Re: What I wish I knew when I became CTO

#185

Post by afarrell »

blackflame7000 wrote: post #141
latch wrote: post #65

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniq…

Testing is absolutely critical don't get me wrong, but you can't test what you can't predict, so there needs to be a distinction between tests that really stress the system in unknown ways vs verifying your ADD function did indeed add N consecutive numbers correctly.

> verifying your ADD function did indeed add N consecutive numbers correctly

This kinda depends on if it is a public or private (or `__private(self):`) method. If its private, no need to test it. But suppose that rather than using something in an existing library, you are bothering to write your own ADD function and expose it to the rest of your codebase. Wouldn't that indicate that your function was special enough that it should be tested?

Re: What I wish I knew when I became CTO

#186

Post by aianus »

ghayes wrote: post #81
aianus wrote: post #76

Earlier quoted context omitted.

> How many tests does a PR need? One? Five? Enough so the overall coverage doesn't go down.

Be careful that coverage is a proxy metric to good tests. Striving for high coverage can mislead you on the quality of your tests.

High coverage is necessary but not sufficient, sure. I don't think you can have a good test suite with low coverage (< 80%).

Re: What I wish I knew when I became CTO

#187

Post by saguro »

arekkas wrote: post #176
forgotpw1123 wrote: post #146

Earlier quoted context omitted.

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Let's break this down. > I'm convinced that unit tests don't usually find bugs. They don't, they test whether or not the API contract the developer had in mind is still valid or not. > IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. You don't write test to find bugs (in 98% of cases), but you can write tests for bu…

There's a much easier way to break it down.

Tests are a pattern. And patterns are the bread and butter of the medicore. That's not to say that patterns or tests are bad, but high calibre guys know when to use which tool. As a tool, unit testing is almost useless.

Low calibre guys don't have any feel for what they're doing. They just use the tools and patterns they were taught to use. All the time. This goes from engineers to managers to other disciplines.

I've seen people at a factory floor treating my test instructions for a device I built as some kind of bible gospel. I had a new girl who had no idea I designed said gadget telling me off for not doing the testing exactly like the instruction manual I wrote says.

The same thing happened with patterns and unit tests. You have hordes of stupid people following the mantra to the letter because they don't actually understand the intent. Any workplace where testing is part of their 'culture' signals to me that its full of mediocre devs who were whipped into some kind of productivity by overbearing use of patterns. It's a good way to get work done with mediocre devs, but good devs are just stifled by it and avoid places that force it.

Re: What I wish I knew when I became CTO

#188

Post by afarrell »

mkarazin wrote: post #13

> I’ve found it a real struggle to get our team to adopt writing tests. I find this hard to believe. Do others CTOs / team leads find this to be the case? I've been a CTO of two small startups with 3-7 developers. We've had resistance to tests at some points (myself included). We've solved it fairly simply. All pull requests (PRs) require tests. PRs are rejected immediately without tests. If a PR doesn't have tests a…

As an engineer, I would now be very wary of joining any company unless "Do they write tests for their code?" as part of the hiring criteria. If you want to have something be part of your culture, it needs to be part of the judgement exercised by humans in the hiring/performance evaluation process. I say "by humans" because you do need someone exercising actual judgement rather than checking a box.

What you outline seems reasonable, at least in an environment where you sometimes have hard deadlines (eg. Ticket sales for this festival go live next week). Outside of that, I'm curious what cases there are where you can have a PR which is both critical to merge and doesn't need tests. When I review a PR, I look at the tests as one way of thinking through "what edge cases have already been accounted for here?"

Re: What I wish I knew when I became CTO

#189

Post by blackflame7000 »

chatmasta wrote: post #177
siliconc0w wrote: post #97

So hiring is pretty hard but I kinda disagree with most of the points there.. * only hire when desperate Strong talent is so hard to get you should probably always be hiring. If you're hiring too many people your bar is probably too low. * only hire to keep up with growth You need to be at least a little preemptive. The hiring process itself can take months, plus the time to train even good new hires is at least a fe…

> Don’t hire someone to do something you’ve not yet figured out I think this is not an indictment of hiring for something you do not know how to do, so much as it is of hiring someone before you have a defined job for them to do. When you’re hiring an engineer, presumably you’ll be placing them onto a team that is responsible for some well-defined part of the stack. So you should know what skills you’re looking for w…

The Whiteboard is nothing more than a hazing ritual testing marathon runners on their 100-yard dash. As CTO I opted to go for two-pronged:

1) give them a take-home project in an area relating to the position they want to weed out the unqualified.

2) bring them on site and speak with them in persona along with other members of the team they will be joining.

3) Its fairly easy to tell whos a whos an impostor if you are knowledgeable yourself, but a group of engineers can identify a faker fairly quickly.

4) Always consult your team about the new hire and don't make it unilaterally or their failures will reflect on you. Even their success won't make up for it if they turn out to be a nutjob and you vouched for them.

Re: What I wish I knew when I became CTO

#190

Post by tomp »

crdoconnor wrote: post #180
latch wrote: post #151

Earlier quoted context omitted.

Thanks. I also have a history with tests and I continue to struggle to find the right balance not just between coverage, but also unit vs integration (and within unit, between behavioural and implementation). I think this uncertainty based on experience is in a whole other class than "I can't get my employees to write tests." Two quick points. 1 - I've added fuzz testing to my arsenal and find it a good value, especi…

>That said, testing is primarily a design tool (aka, identifying tight coupling). The more you do it, the more you learn from it, the less value you get from it because you inherently start to program better Unit tests "identify" tight coupling because they are themselves a form of tight coupling.

Huh? My interpretation is, it's harder to write shitty code (e.g. hard-coding the database server IP) if you write unittests (where you'll need to abstract the database interface to be able to mock it). In this manner, unittests promote clean, separated interfaces and work against tight coupling.
Post reply on HN