Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Servers Are Burning

logicmag.io

Post reply on HN
Print view View original

131–140 of 158 posts

Re: The Servers Are Burning

#131

Post by spc476 »

viraptor wrote: post #102
tekstar wrote: post #56

Earlier quoted context omitted.

What's an example of a memory leak that can't be found by valgrind?

One happening in an embedded language with its own GC. Unless you can switch all allocations to explicit management for the tests, it's almost impossible to differentiate between live objects, free lists, cycles with deferred collection, and arenas that are supposed to be freed on exit.

That happened with me at work. A certain class of error would leak references and over time they would pile up. Fortunately, Lua (the embedded language in use) allows one to mark certain references as weak references and allow collection. The trick was in determining what to mark as weak references.

Re: The Servers Are Burning

#132

Post by spc476 »

evancox100 wrote: post #114

"If you wrote buggy software, why would the software you wrote to check that software be any less buggy?" I work in chip design, where bugs can be rather costly, e.g. $1mm for a new mask set, not to mention the months it takes to get back new hw. The situation described above is why we try to have one person do the design and a different person do the verification/testing. A lot of the time, the test writer will trea…

When I was doing QA at work, I never used the official tools to generate the data files required for testing the application, but wrote my own. This brought to light a ton of issues and unwritten assumptions (both in my code, and in the application).

Re: The Servers Are Burning

#133

Post by LandR »

megablast wrote: post #112
toss1 wrote: post #84

Earlier quoted context omitted.

Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

You are acting like most bugs aren't already caught in the dev process. No on writes code and just deploys. Writing extra tests that cover everything for a small project that you are working on yourself is a waste of time.

> No on writes code and just deploys.

Oh yes they do!

Hell, I've seen dlls copied from a server, disassambled into code, code changed, rebuilt and copied back to server....

Most devs here write code then copy their bin folder to a production server and deploy it.

Re: The Servers Are Burning

#134

Post by spc476 »

nradov wrote: post #113
always_good wrote: post #105

Earlier quoted context omitted.

Only if you are referring to perfect tests which cost zero time to write/maintain and only the tests written that eventually will catch an issue. Otherwise, tests have a cost just like any other code. You can see this by looking at both extremes: perfect tests (described above) and useless tests. For example, tests that make your codebase too brittle, tests that don't actually test anything useful, spending too much…

Tests save you time. They have a negative cost.

The regression test [1] for our legacy application takes five hours to run. It takes five hours to run because it also needs to check every log message (via syslog) to ensure every transaction happened.

I can't say it saves any time, and it certainly does not have a negative cost.

[1] There are no unit tests, as there are no real "units" to test. It's a legacy code base of C and C++, using a proprietary library that no one left in the company has much experience with. Said proprietary library is considered "legacy" by the company that owns it. I've seen the code. No wonder they consider it "legacy" (it started life in the mid-80s and god does it show).

Re: The Servers Are Burning

#136

Post by dawidw »

mikestew wrote: post #64
fhood wrote: post #40

Earlier quoted context omitted.

You can't possibly say that with any confidence. I literally just finished reproducing a memory leak caused by an insane combination of circumstances. This was not something valgrind or any similar tool would find. It was not something that any rational human being would have thought to write tests for. Once code is no longer synchronous, and tasks are being juggled around and swapped between, it becomes virtually im…

You can't possibly say that with any confidence In answer to the original question, I stand behind my answer with 100% confidence. The test infrastructures I build catch memory leaks all of time, and I do not consider it abnormal for them to do so. If you mean to say that I cannot be confident that I have caught all of the memory leaks, well duh, of course I can't. But that's not what was asked, and that's not what I…

> The test infrastructures I build catch memory leaks all of time [...]

Could you elaborate on that, please? I'm curious. Thank you in advance!

Re: The Servers Are Burning

#137

Post by berti »

shady-lady wrote: post #99
chrisweekly wrote: post #96

Earlier quoted context omitted.

Thanks for the rec! Just bought for $5 (kindle ed).

what site? i can only see manning for $36. not on amazons.

I'm guessing this one (with a slightly different title: Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing) http://a.co/d/6tGlrUc

Re: The Servers Are Burning

#138

Post by davedx »

annywhey wrote: post #127
rstuart4133 wrote: post #116

Earlier quoted context omitted.

My philosophy is a little different. It grew out of writing a thesis on software quality in a post grad course. We studied lots and lots of ways to improve software quality. I can't count the number of papers I read on the subject. The first paper you read is all gung-ho about their particular method of course so you think you are done - you've found the holly grail. Then you find the second paper is equally gung-ho…

Although I wouldn't argue against unit testing(I use it too), I'm a little more hesitant about incentivizing test quantity. Many times when this comes up having been tried it gets attached to a anecdote that goes: "And so we wrote very bad tests and atomized the codebase into unmaintainable pasta code that could be trivially tested to make the number go up." Which leads me to a hypothesis that there's some other quan…

I agree with this, and I think software quality is like security. Multiple layers are better, and orthogonal layers are even better still.

A good logging system is a great example. To be effective, your logging should be going into something like Kibana or Grafana or both. Logging levels should be correct. Errors should be easily visible and have useful stack traces; all other noise should be easily filtered out. Alerting should be in place for any crashes, ideally going directly into a bug tracking system. (Sentry.io has a nice story for this on the front end).

This ties in well with the "let it crash" philosophy. You just cannot anticipate all the crazy inputs that will go into the typical software system of today. You can't. Most systems I work with have multiple external dependencies, each with their own schemas, data workflows, deployment processes and so on. Unit testing won't be as valuable with that stuff. Full end-to-end testing will be more useful.

Look at all your options as a whole and allocate developer resources according to the value of each.

And personally, I don't think the OkCupid model is as bad as it sounds... ;)

Re: The Servers Are Burning

#139

Post by rhencke »

megablast wrote: post #112
toss1 wrote: post #84

Earlier quoted context omitted.

Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

You are acting like most bugs aren't already caught in the dev process. No on writes code and just deploys. Writing extra tests that cover everything for a small project that you are working on yourself is a waste of time.

> No on writes code and just deploys.

I wish this was true.

I stopped keeping track of how many times I've seen people break stuff because they deploy code that simply doesn't work, because they figured they could just eyeball it rather than run it and test it out.

Not everyone cares deeply about the quality of their work.

Re: The Servers Are Burning

#140

Post by tinco »

sidstling wrote: post #121
gred wrote: post #75

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-sce…

Don’t you take this bit about out of context? If the premise the article builds around faulty tests is correct, then stuff like unit-testing is mostly a waste of time since you’ll still need to fix things when the servers are burning. We’ve done TDD, projects with coverage only vital parts and no software-tests where I work and there is no difference in production on smaller projects. Maybe we’re terrible at writing…

I think the advantages of tests, TDD or otherwise, are not easily measured. Everyone knows cowboy coding up small projects works and through manual testing they can quickly be deployed with reasonable quality, especially if the developers are experienced.

The tests shine in the follow-up phases, when code needs to be changed, when inexperienced hands have to touch the code base, and when there's simply too much going on to fit in one mind.

If you build your product with the future in mind, then you write tests. If you've got VC money and you need to sprint to some goal before the puck gets in front of your stick, then I can see the logic in skipping tests.

If the tech debt hits you before you reach that point though, you're in deep shit. It might also be the reason that innovative companies often seem to technologically stall right after hitting mainstream.

Post reply on HN