Live data from Hacker News

Ask HN: How to convince big tech team that tests and code quality matter?

news.ycombinator.com

51–60 of 76 posts

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#51

Earlier quoted context omitted.

Totally agree with all your points except >there's not a lot of empirical evidence to support claims that automated tests will improve developer productivity or code quality. If you'd have said "there aren't any published papers _proving_ this" I would've agreed simply because I do not know if such studies exist or not (they might); but I'd argue that automated tests obviously improve code quality and productivity. A…

I'm replying to this and also some of the sibling comments. No, automated development-time tests do not necessarily equate to higher quality, given that they: - Take time away from other activities - May provide a false positive, lulling developers into a false sense of confidence - Increase the time taken for certain CI/CD operations, sometimes by hours or days! - The presence of a large volume of "test" code means…

Types do a great job at assuring you’re getting generally the right kind of data in and out of different functions/methods, but they don’t assure that the program is actually doing the right thing. Strong typing just helps to assure that the program isn’t doing specific categories of the wrong thing (in the same way that Rust and Go make it difficult to write memory-unsafe code). I mean, Go and Rust both have testing capabilities built in. If it wasn’t necessary, why would they do that?

Beyond the most trivial of projects, I can’t think of a single project that I’ve worked on (or even looked at on GitHub) that has weighed the costs and benefits of automated testing and decided it just wasn’t worth it.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#52
post #49
post #6

Aside from what Flankk suggested about being one with the tribe: while you're waiting for your tribal acceptance card, measure the problem in silence. The change defect rate (how many percent of your commits are made to fix a problem introduced in an earlier commit) is very easy to sample manually, and gives you an idea of the scale of the problem. Then you can go to the team and say, "Look, 40 % of the things we do…

But no one does anything like you mentioned. OP already has his mind set: *This must lead to a lot of problems which I'll only face after ramping up, but even onboarding is painful if you don't have e.g. clear automated testing set up.* He just KNOWS that it MUST lead to a lot of problems. I am fighting with this mindset at my work and I am sick and tired because people who KNOW - don't show any measurements. Even if…

One of my favourite sayings:

> It ain't what you don't know that gets you into trouble; it's what you know for sure that just ain't so.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#53
post #19

It takes a little time for people to see why testing is important. Since you're new to the organization, tread lightly and keep an eye out for places where testing can really make everyone's life better. I've found that there are a few key junctures at which people take to testing. Here are some: 1) When dead-simple unit-tests catch old bugs. If cos(0) doesn't return 1, there's a problem. 2) When unit tests, for the…

> some testing is worlds better than no testing

While it is probably true most of the time, like many things in live, "it depends".

When you are developing some not so well defined functions and everything is on quicksand, tests may just slow your progress, while not really providing useful feedback (i.e.: when you always modify a function and its test in tandem). Ideally you should clarify your requirements, but sometimes the process of producing the code is the refining step

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#54

Earlier quoted context omitted.

I'm replying to this and also some of the sibling comments. No, automated development-time tests do not necessarily equate to higher quality, given that they: - Take time away from other activities - May provide a false positive, lulling developers into a false sense of confidence - Increase the time taken for certain CI/CD operations, sometimes by hours or days! - The presence of a large volume of "test" code means…

Types do a great job at assuring you’re getting generally the right kind of data in and out of different functions/methods, but they don’t assure that the program is actually doing the right thing. Strong typing just helps to assure that the program isn’t doing specific categories of the wrong thing (in the same way that Rust and Go make it difficult to write memory-unsafe code). I mean, Go and Rust both have testing…

Conversely, I've actually never seen automated tests "in the wild". Every project I've come across has had zero tests, or close to it.

I'm dealing with a migration of a legacy code base right now, today, that could probably use some tests! But even suggesting this is a complete non-starter: The time required to write the tests vastly exceeds the time and budget allocated to the migration project. I'll simply enable an APM tool that costs $50 a month and call it a day. If something crashes in UAT we'll catch it. If not, we'll definitely catch it in PRD. If not, then it's probably not worth dealing with!

I've added tests to some of my projects in the past and caught virtually nothing with them. I always run my code "through its paces" at least a few times with a REPL, debugger, or some sort of tracing tool before it's ever committed to the repo.

The kind of bugs I find in production would never be found by typical automated test suites. Things like logic errors due to misunderstanding the requirements, or deadlocks in a database that only manifest with an obscure multi-user workflow combination.

As a random example, a significant issue in one codebase was that a distinct sort was case-insensitive but should have been case-sensitive. It was dropping a few hundred items out of hundreds of millions. Will your tests find this issue? What if I told you nobody noticed for years, and that the processing where this manifested takes 4 hours?

Will your dev team still be productive if changes require 4-hour tests every time they hit build or check in the code? To find a one-time issue that won't reoccur now that it's been fixed? Really?

Small, fast tests often find nothing and are worthless because they don't test anything of interest.

Big complicated tests are time consuming to write, still often find nothing, and slow down processes.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#55
Formal style guidelines and mandatory tests for everything are largely wastes of time and often detrimental to software development.

To elaborate: formal style guides prevent simple things like formatting code in a manner that is easier to read in some cases (“why is this newline here? Why did you column aligned this block of expressions?”), and more insidiously force you to write worse code because they constrict you. Style guidelines should be just that - guidelines and engineers should have the decency to not request stylistic changes during code reviews unless they spot code that is obviously sloppy.

As for tests: insisting on having tests for everything impedes development in the present as well as in the future. More crucially the compulsive desire to have everything testable makes you write worse code as you need to abstract away parts that would simply be function calls or use patterns that obfuscate the code. Some will claim that this results in better designed code, but realistically speaking it just results in more complex code, which is almost always worse. You don’t need to use abstractions everywhere. Developers should focus on making shit work well and not over engineering code because they want to feel smart all the while making rationalizations about testable code and whatnot. I digress though so back to my original point regarding tests: in order for tests to justify their existence they have to test something significant and/or test a module that provides a service (read: hidden behind well defined API, api which you test to ensure that it doesn’t break its “contract”)

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#56
See Chesterton's fence: "The principle that reforms should not be made until the reasoning behind the existing state of affairs is understood." [https://en.wiktionary.org/wiki/Chesterton%27s_fence]

There might not be a particular reason for why things are the way they are. It might have just grown to be this way.

I haven't read it but I've heard good things about "Working effectively with legacy code" by Michael Feathers.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#57

Earlier quoted context omitted.

Types do a great job at assuring you’re getting generally the right kind of data in and out of different functions/methods, but they don’t assure that the program is actually doing the right thing. Strong typing just helps to assure that the program isn’t doing specific categories of the wrong thing (in the same way that Rust and Go make it difficult to write memory-unsafe code). I mean, Go and Rust both have testing…

Conversely, I've actually never seen automated tests "in the wild". Every project I've come across has had zero tests, or close to it. I'm dealing with a migration of a legacy code base right now , today, that could probably use some tests! But even suggesting this is a complete non-starter: The time required to write the tests vastly exceeds the time and budget allocated to the migration project. I'll simply enable…

I worked on a project with no tests. We had to refactor a specific functionality changing database entities, augmenting algorithms and adding behaviour. Hundreds of possible cases, no way to be tested manually. I refactor all the code to be "testable", I wrote hundreds of tests. I wouldn't have done it without tests. In the same project, a single algorithm had a dozen of parameters, dozens of possible scenarios. An issue occured in prod. Luckily algorithm had over 60 tests, we added a red test, fixed, run all green and we were very confident of the release, same confidence not possible without tests.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#58
This is tough. It all begins with the person leading the team. Even as CEO, I had a tough time convincing my CTO to do this. There are only two ways I see this could change - either the leadership changes or something drastic happens which forces them to understand the importance of this. Mostly the former is what is going to pan out IMO. People rarely change

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#59
I fought exactly this battle for two years on my second team at Facebook. Never got very far TBH. Part of the reason is the "become a member of the tribe first" thing that others have mentioned, but there's a kicker: any variance from the existing attitude about tests will itself tend to make you an Outsider. Here are some suggestions for things you can do to gently and slowly nudge things toward a better place.

  * Make sure your own changes are properly tested.

  * Make sure existing tests are reliable and quick to run, since these are common excuses for not doing more.

  * Develop tooling to make writing tests easier, or to write new kinds of tests. Key word is "new" because that taps into the neophilia that's endemic at these places.

  * Use new techniques to reliably reproduce bugs that are still fresh in people's minds, and to show how this can find related bugs as well.
Don't spend too much time on any of these things, though. That will reduce your recognized "impact" and slow the process of gaining trust from the tech-lead "guardians of culture" who have usually decided (based on a mere 5-10 years' experience at only one company) that their current approach is perfect. Better to maximize that impact and then start promoting change. So, to a large extent, "drink the Kool Aid and shut up" really is the best approach available.

Re: Ask HN: How to convince big tech team that tests and code quality matter?

#60

See Chesterton's fence: "The principle that reforms should not be made until the reasoning behind the existing state of affairs is understood." [ https://en.wiktionary.org/wiki/Chesterton%27s_fence ] There might not be a particular reason for why things are the way they are. It might have just grown to be this way. I haven't read it but I've heard good things about "Working effectively with legacy code" by Michael Fe…

I'm a huge fan of citing Chesterton's Fence, but it's much more applicable to a presence than to an absence. When something is missing the most common reason is sloth, and that doesn't deserve the same detailed consideration as when something's there.
Post reply on HN