Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What I wish I knew when I became CTO

medium.com

Post reply on HN
Print view View original

151–160 of 258 posts

Re: What I wish I knew when I became CTO

#151

Post by latch »

forgotpw1123 wrote: post #146
latch wrote: post #65

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniq…

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Thanks. I also have a history with tests and I continue to struggle to find the right balance not just between coverage, but also unit vs integration (and within unit, between behavioural and implementation). I think this uncertainty based on experience is in a whole other class than "I can't get my employees to write tests."

Two quick points.

1 - I've added fuzz testing to my arsenal and find it a good value, especially if you're blasting through a new project.

2 - Good monitoring (logging, metrics) trump tests when it comes to quality, both for being _much_ easier to do and in terms of absolute worth.

That said, testing is primarily a design tool (aka, identifying tight coupling). The more you do it, the more you learn from it, the less value you get from it because you inherently start to program better (especially true if you're working with static typed languages, where coupling tends to be more problemantic). There are secondary benefits though: regression, documentation, onboarding.

I think one key difference between what you describe and my own situation is that my small team manages a lot of different projects. Maybe their total size is the same as your two, but our ability to confidently alter a project we haven't looked at in months is largely due to our test coverage. I agree that then and there, I get less benefits from tests for the project that I'm currently focused on.

The nut I haven't cracked yet is real integration test between systems. This seems like something everyone is struggling with, and it's both increasingly critical (as we move to more and and more services) and increasingly difficult. My "solution" to this hasn't been testing, but rather: use typed massaging (protocol buffers) and use Elixir (the process isolation lets you have a lot of the same wins as SOA without the drawbacks, but it isn't always a solution, obviously)

If you're interested, I've written more about this: http://openmymind.net/A-Decade-Of-Unit-Testing/

Re: What I wish I knew when I became CTO

#152

Post by latch »

sagichmal wrote: post #145
latch wrote: post #90

Earlier quoted context omitted.

My experience is that what you're saying is true if it's being done dogmatically. I've been a subject (victim?) of this before. But strategically applied? I think it's a pretty big win. Specifically, I'm talking about onboarding people (onto the company or a project) and working with interns and juniors. Doesn't even have to be a senior and a junior, two juniors working together is significant. And it isn't just abou…

> I'd say at a minimum, having everyone spend 4 hours a week pair programming should be a goal to try. Pair programming is like nails on a chalkboard to me and at least a plurality of developers generally, based on what I’ve experienced personally and read online. An expectation that I’d do 4 hours a week of it would have me hunting for a new job immediately. It’s different in kind to other practices like mandatory c…

Ok, so that assertion was pretty controversial, but, honest question, what mechanism do you use for mentoring/learning/growth? Code review is the only other activity that I've seen that can have the same type of impact, but I see them as complementary.

I'm old, I learned about sql injection and hash salts and coupling and testing by being awful at it for decades. How do I transfer that knowledge so that a 26 year old can be where I was at 32 if not working closely with them, using their code as the foundation for that knowledge transfer?

Re: What I wish I knew when I became CTO

#153

Post by fsloth »

fergie wrote: post #150
latch wrote: post #65

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniq…

To be fair, testing is to some extent an unsolved problem. The joys of testing were being extolled long before test frameworks were actually usable. Now that they are, and you can glue Travis, GitHub and the test lib of your choice together pretty easily you have solved about 30% of what needs to be tested. If, say, you are developing an Office add-in on a Mac, and you want to test it on Word 2013 on Windows 7, there…

I'd say that is an architecture problem up to a point. A test does not need any framework. Simply a defined output for a defined input, and then check whether the output matches expectations.

Re: What I wish I knew when I became CTO

#154

Post by guelo »

watwut wrote: post #149
forgotpw1123 wrote: post #146

Earlier quoted context omitted.

PR comments I agree with, but after believing in unit tests for years I'm drifting slowly into the "waste of time" camp. I'm convinced that unit tests don't usually find bugs. IMO, most bugs are edge cases that were an oversight in the design. If the dev didn't handle the case in code they're not going to know to test for it. Fuzzing is a much better approach. At my current position I have the opportunity to work wit…

Unit tests are not meant to find bugs, they are meant to keep it together as you add features and refactor the system. They are also completely orthogonal to patterns and layers and aspects and J2EE and what not. All that has nothing to do with tests at all.

I think it can be related in a certain type of poor team that cargo cults strict rules and patterns and mechanistically writes tests for every little getter and setter of all the useless layers of useless glue classes whose real purpose is to mask their lack of understanding.

Re: What I wish I knew when I became CTO

#155

Post by sagichmal »

latch wrote: post #152
sagichmal wrote: post #145

Earlier quoted context omitted.

> I'd say at a minimum, having everyone spend 4 hours a week pair programming should be a goal to try. Pair programming is like nails on a chalkboard to me and at least a plurality of developers generally, based on what I’ve experienced personally and read online. An expectation that I’d do 4 hours a week of it would have me hunting for a new job immediately. It’s different in kind to other practices like mandatory c…

Ok, so that assertion was pretty controversial, but, honest question, what mechanism do you use for mentoring/learning/growth? Code review is the only other activity that I've seen that can have the same type of impact, but I see them as complementary. I'm old, I learned about sql injection and hash salts and coupling and testing by being awful at it for decades. How do I transfer that knowledge so that a 26 year old…

I like embedding, joint design sessions, and thorough code review. For explicit junior dev mentor relationships, I like frequent one-on-ones (I’ve even done 2x per week) and quite detailed, joint ticket/work planning. I’m also happy to do pair code analysis/review for areas that I’m familiar with and the junior isn’t.

What I’m not happy to do, and what pair programming is, as far as I have seen, is to sit down with another engineer and figure out a problem together. In addition to being simply incompatible with my mind’s problem-solving faculties, it in my experience produces lowest-common-denominator software that strips any sense of self-satisfaction or -actualization from my work. No thank you.

Re: What I wish I knew when I became CTO

#156

Post by latch »

sagichmal wrote: post #155
latch wrote: post #152

Earlier quoted context omitted.

Ok, so that assertion was pretty controversial, but, honest question, what mechanism do you use for mentoring/learning/growth? Code review is the only other activity that I've seen that can have the same type of impact, but I see them as complementary. I'm old, I learned about sql injection and hash salts and coupling and testing by being awful at it for decades. How do I transfer that knowledge so that a 26 year old…

I like embedding, joint design sessions, and thorough code review. For explicit junior dev mentor relationships, I like frequent one-on-ones (I’ve even done 2x per week) and quite detailed, joint ticket/work planning. I’m also happy to do pair code analysis/review for areas that I’m familiar with and the junior isn’t. What I’m not happy to do, and what pair programming is, as far as I have seen, is to sit down with a…

Thanks, I appreciate the perspective.

Re: What I wish I knew when I became CTO

#157

Post by merb »

fergie wrote: post #150
latch wrote: post #65

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniq…

To be fair, testing is to some extent an unsolved problem. The joys of testing were being extolled long before test frameworks were actually usable. Now that they are, and you can glue Travis, GitHub and the test lib of your choice together pretty easily you have solved about 30% of what needs to be tested. If, say, you are developing an Office add-in on a Mac, and you want to test it on Word 2013 on Windows 7, there…

well it happens a lot of techniques are still not understood.

I mean we develop database heavy code. Should we never test the code running with the database? would be a poor choice since we would loose a lot of coverage. What we did instead were transactional tests. Which means that in PostgreSQL sense that we actually use SAVEPOINTS to actually wrap our tests inside a savepoint and than rollback to the sane state and never commit anything to the database. With DI this is fairly easy since we can just replace the database pool with a single connection that uses the pg jdbc driver which can insert these savepoints.

Test suite runs ~4 minutes (scala full compile + big test suite ~65%+ coverage (we started late)) in best cases and can be slow if we have cache misses (dependencies needs to be resolved, which is akward slow on scala, sbt)

Re: What I wish I knew when I became CTO

#158

Post by krzrak »

ecshafer wrote: post #117
speby wrote: post #110

Earlier quoted context omitted.

In a word, no. But in startup-land, where the total number of people on the engineering team is, say, less than 10, chances are good that the CTO will also play a lead engineer and/or architect sort of role, in which case they will play a part in designing the architecture, selecting frameworks, and so forth. CTO of, say, US Foods? No, of course not.

Why call the role a CTO then? If the role is closer to a tech lead or architect, just call them an architect. This has always confused me in start up land. There will be a full c suite in a company of 10 people, despite that those c suite folks day to day would look nothing like a corporate position. Just call it what it is instead of inflating titles.

> Why call the role a CTO then? If the role is closer to a tech lead or architect, just call them an architect.

I agree! How many times I got job offers that read basically: "CTO/sole developer". That's inflated and meaningless, like being one of 100 Senior VPs in a bank.

Re: What I wish I knew when I became CTO

#159

Post by millstone »

fsloth wrote: post #153
fergie wrote: post #150

Earlier quoted context omitted.

To be fair, testing is to some extent an unsolved problem. The joys of testing were being extolled long before test frameworks were actually usable. Now that they are, and you can glue Travis, GitHub and the test lib of your choice together pretty easily you have solved about 30% of what needs to be tested. If, say, you are developing an Office add-in on a Mac, and you want to test it on Word 2013 on Windows 7, there…

I'd say that is an architecture problem up to a point. A test does not need any framework. Simply a defined output for a defined input, and then check whether the output matches expectations.

In this sort of scenario, the bugs lie in the expectations themselves. Tests that don’t account for that are dead weight.

Re: What I wish I knew when I became CTO

#160

Post by auganov »

ereyes01 wrote: post #99
Rapzid wrote: post #73

Earlier quoted context omitted.

How many tests does a PR need? One? Five? When would you not write tests for something(that something not necessarily being a PR, but maybe a unit or feature)? Tests, like any process, should be serving you and your goals.. You shouldn't be serving your processes or testing practices. This sort of un-nuanced thinking isn't indicative of a high performing startup or CTO IMHO. Perhaps your policies are not directly ind…

I guess it might sometimes be fine to be a relativist and write off the need for tests as a result of "nuanced thinking," but I think you have to accept that you are running a risk by shipping untested code into your product. As others have said, line coverage is a misleading metric. Ideally, your tests would fully cover all _program states_, and even 100% line coverage doesn't guarantee full state coverage. If you h…

Code without test code doesn't mean untested code. And vice-versa.

Unless you're actually writing complex tools - no, you're probably not getting a "formalized way of modeling" what happens to your program.

If somebody tells me "hey, I have to keep manually testing this and that, I'm losing a lot of time, how about I spend 2 days writing my test thing?" - I'll say Sure!

But if someone tries to convince me in the abstract - I'll be skeptical. Developer busy-work is real.

Post reply on HN