Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Servers Are Burning

logicmag.io

Post reply on HN
Print view View original

121–130 of 158 posts

Re: The Servers Are Burning

#121

Post by sidstling »

gred wrote: post #75

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-sce…

Don’t you take this bit about out of context? If the premise the article builds around faulty tests is correct, then stuff like unit-testing is mostly a waste of time since you’ll still need to fix things when the servers are burning.

We’ve done TDD, projects with coverage only vital parts and no software-tests where I work and there is no difference in production on smaller projects. Maybe we’re terrible at writing tests, but if so, then it’s still something we have to deal with.

So I think it’s hard to outline the correct amount of testing for every case.

I don’t think I would want to work on a major system with multiple contributors if there weren’t automatic tests, so it’s not like I’m against testing either, I just don’t think it’s always a holy grail.

Re: The Servers Are Burning

#122

Post by flukus »

aLifeBot wrote: post #51
mikestew wrote: post #19

Earlier quoted context omitted.

Mine do.

Which framework do you use?

Not OP but make. It's not exactly a framework but I've got a project where every test get's run twice, first as a test to check the functionality, then run under valgrind.

It won't catch all possible errors but it will catch most. On the plus side it will only nag about actual errors, not potential errors like "safe languages".

Re: The Servers Are Burning

#123

Post by mirimir »

I'm reminded of the day that I discovered SQL cross joins. Some decades ago, working in Access on WinXP. So I executed, and nothing much happened. So I went for lunch, and when I got back, the machine was frozen. So I went WTF?, and hard reset. And then did it again, a few times.

And the cool thing was that doing that, and running out of disk space, apparently didn't damage the system. Microsoft must have designed WinXP to tolerate that.

Re: The Servers Are Burning

#124

Post by __david__ »

ozim wrote: post #65
3pt14159 wrote: post #41

I appreciated this article because I so infrequently get to hear from people that hold this view of testing, which I do not share. To me, the importance of tests are a function of the consequence of failure and the likelihood that failure will happen. It's a question of hazard and risk. One thing to realize is that code that lasts longer is more likely to fail because the people and libraries that support it are pron…

I think just SSHing to box and doing deploy is not function of users as you noticed but function of how many devs are working on project. For 1-2 devs manual deploy might be ok, when 3rd comes in, it is time to get rid of any magic that might happen when deploying by hand. Because you know someone will do something special and not document that, when he updates automated deploy he does not have to document or talk wi…

It's also a function of how many servers you have. If you have to ssh into more than one box to deploy then you should probably have automation, even if you only have an ops team of 1 or 2.

Re: The Servers Are Burning

#125

Post by aprdm »

megablast wrote: post #112
toss1 wrote: post #84

Earlier quoted context omitted.

Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

You are acting like most bugs aren't already caught in the dev process. No on writes code and just deploys. Writing extra tests that cover everything for a small project that you are working on yourself is a waste of time.

> No on writes code and just deploys.

You would be - very - surprised how some big non tech companies (FANG) run.

Re: The Servers Are Burning

#126

Post by cs02rm0 »

nitrogen wrote: post #101
toss1 wrote: post #84

Earlier quoted context omitted.

Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

The logic kind of makes sense if you consider that customers will only find a small fraction of the bugs that your team will. Sometimes the cost of removing all bugs is greater than the cost of losing customers to the few unlucky bugs that get found. But you really need experienced people deciding where that tradeoff lies for each company and project.

I think that's the bottom line.

I've done virtually zero unit testing on a project before when there was barely time to write the core code. There wasn't much business logic, lots of I/O and no user inputs or browsers to worry about and I was leading a small team of four so it was relatively well suited to it. Still, I was nervous as it was quite high profile and unit test coverage was about the only metric management knew to ask for.

It was replacing a critical third party system being turned off on a fixed date and we barely made it. 6 months after being switched on in production we'd had zero bugs reported and I moved on. I'm confident in hindsight that I made the right call for the circumstances, but having it funded properly would have made me sweat a bit less.

Re: The Servers Are Burning

#127

Post by annywhey »

rstuart4133 wrote: post #116
bunderbunder wrote: post #44

> in order to write effective tests, a programmer had to know all of the ways that a piece of software could fail in order to write tests for those cases No. In order to write effective tests, a programmer has to think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes. W…

My philosophy is a little different. It grew out of writing a thesis on software quality in a post grad course. We studied lots and lots of ways to improve software quality. I can't count the number of papers I read on the subject. The first paper you read is all gung-ho about their particular method of course so you think you are done - you've found the holly grail. Then you find the second paper is equally gung-ho…

Although I wouldn't argue against unit testing(I use it too), I'm a little more hesitant about incentivizing test quantity. Many times when this comes up having been tried it gets attached to a anecdote that goes: "And so we wrote very bad tests and atomized the codebase into unmaintainable pasta code that could be trivially tested to make the number go up."

Which leads me to a hypothesis that there's some other quantifiable out there that would work better than unit test coverage: a "diversity test" metric. The software artifacts produced by writing code and after a build are not "the software" in the sense of the whole project. Documentation, support, and so on are also "the software." Logging and profiling tools enable debugging methods other than "stare at the code very intensely". If you start from a holistic standpoint and assume quality requires multiple angles of inspection then your choices of technique broaden substantially.

If you changed a mandate of "must test all code" to a mandate of "must either test, log, profile, or document all code" you would add leeway for the corners that are test-averse to be documented or logged instead, and vice-versa. The combination gives you a palette of feedback: if it's hard to do any of those things, you have bad code. Ideally you can get all of it, but that is unlikely to be the norm for most organizations most of the time. But more bugs will be caught by this web of feedback than if you mandate any one of those methods alone.

Re: The Servers Are Burning

#128

Post by leoedin »

nradov wrote: post #113
always_good wrote: post #105

Earlier quoted context omitted.

Only if you are referring to perfect tests which cost zero time to write/maintain and only the tests written that eventually will catch an issue. Otherwise, tests have a cost just like any other code. You can see this by looking at both extremes: perfect tests (described above) and useless tests. For example, tests that make your codebase too brittle, tests that don't actually test anything useful, spending too much…

Tests save you time. They have a negative cost.

Some tests save you time. Some tests don't. It's perfectly possible to write a huge, fragile, heavily coupled test suite which takes days or weeks to modify when you make even a tiny change to the code it's testing.

Just as code falls on a spectrum between clean and completely unmaintainable, so do tests.

Post reply on HN