Why Don't Software Developers Use Static Analysis Tools to Find Bugs? • Hacker News Board

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#111

Post by AnimalMuppet » Fri, Jun 26, 2015, 4:11 PM UTC

taeric wrote: post #26

dschiptsov wrote: post #17

Because neither an advanced type system nor static analysis could catch bugs in program logic?) The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

You'd be surprised. Probably at both, but I hasten to get it in the conversation that static analysis is more than just static typing. By a long shot. To me, simple static analysis can be over sold to the point that it is worthless. I swear, I see more effort put into detecting tabs versus spaces than I do things that actually reliably cause bugs. Seriously, unless you are writing make files, I just can't bring mysel…

Or, you used a pointer, then checked whether it was null.

This is why you keep re-running it, by the way. You had your null check at the top of the function, and then in maintenance someone added something new at the top, not realizing that it needed to be after the null check...

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#112

Post by wyldfire » Fri, Jun 26, 2015, 4:12 PM UTC

We do. (some of us do, anyways).

> Mike: "Clang is my favorite. Its built into the compiler. You don't have to invoke anything special."

I agree with Mike. I've also used Coverity Static Analysis and various other tools. Some of them are better than others.

Also, "-Wall -Wextra" or "-Weverything" are fantastic for a simple first-pass static analysis.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#113

Post by karmakaze » Fri, Jun 26, 2015, 5:41 PM UTC

virmundi wrote: post #98

crdoconnor wrote: post #65

Mainly because most of the tool reports freely mix critical issues with stuff that is just opinionated bullshit (e.g. 80 character length lines). When the output is as long as your arm and you have pick through it with a fine toothcomb to find the things that matter (and even they aren't necessarily causes of bugs), the whole idea becomes substantially less appealing.

I'm with you. I used FindBugs within Eclipse, just fine. But on my current project they (the company) tuned Sonar to be extreme. Every opinion is a Major bug at least and breaks the build. I like to have lines like this: Something s = function(....); return s; This allows me to put a break point if I need one in the future. Unfortunately Sonar is configured to say this is a Major bug. It breaks the build until I have…

I found that specific Sonar issue to be bogus. In addition to breakpoints, sometimes the 's' has is meaningful name that ties the function call with the return value.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#114

Post by smarks » Fri, Jun 26, 2015, 5:48 PM UTC

nkurz wrote: post #3

Nice paper from 2013, but I was confused at first how it related to the blog. This paragraph just after the abstract explained, but I missed it the first time through: The original article in PDF[1] was published at the NCSU COE People site. It was translated and published at our blog by the authors' permission. At the end of the article[2], we added a short section about the PVS-Studio analyzer, where we describe wh…

Yes, I also found this quite confusing. The way this paper was attributed seems quite sloppy. It mentions previous publication at the "NCSU COE People site" and gives a link, but omits mention that this paper was originally presented at the International Conference on Software Engineering (ICSE 2013):

http://2013.icse-conferences.org/index.html

These notes shouldn't have been tacked onto the abstract. Instead they should have been clearly separated from the paper itself.

In addition, the PVS Studio team "added a short section" which appears as if it were a modification to the paper, and and it also appears in the augmented table of contents. If one is not reading carefully, or if the section heading has scrolled of the top of the screen, one might be misled into thinking the "added section" was also authored by Johnson et. al. instead of by the PVS Studio team. The "added short section" should also have been published entirely separately.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#115

Post by yeahdude » Fri, Jun 26, 2015, 7:59 PM UTC

crdoconnor wrote: post #83

yeahdude wrote: post #72

Earlier quoted context omitted.

From another perspective, hacking often involves making broken things work better.

From yet another perspective: don't use broken tools.

I keep forgetting this one: don't offer help to complainers.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#116

Post by nachiketn » Fri, Jun 26, 2015, 9:57 PM UTC

Additional work. Developers don't like to deal with false positives. If static analysis tools were like a laser. It's more like a shotgun that throws a bunch of shrapnel. Sure it hits some really difficult to find bugs. But developers don't like to have to go through a long list of warnings that aren't really guaranteed to find a valid bug, especially in the applications' context. For example null checking is good, but is it needed if I know it is never really going to be null? Isn't that arguably extra effort of typing and compiling and couple of extra lines to compute? I think we will truly move to better adoption if: 1) We had better configuration for customizing it for our application. 2) More education among developers about what is the correct way to modify the tool's behaviour to aid their work. Help the tool, help them in return. 3) More tools that do specific and general work. Example tools extremely focused on mobile development but also tools that not just help you with correcting design and coding problems but also security vulnerabilities.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#117

Post by dllthomas » Fri, Jun 26, 2015, 10:11 PM UTC

seanmcdirmid wrote: post #71

danbruc wrote: post #53

Earlier quoted context omitted.

How would maintaining a SSA representation of the code help detecting dead code paths? Doing so requires reasoning about possible truth values of conditions and is in general equivalent to the halting problem and therefore undecidable. You could only spend the cycles available in real time and that will in a certain number of cases be enough to solve the problem but there will also be instances requiring more cycles…

Going incremental is another option: if the analysis was done overnight and you only made a few changes, there shouldn't be much to do to "repair" the analysis according to your changes. But the PL community is not really good with incremental computations, especially ones that must consider arbitrary code changes (which, for example, can cause non-monotonic movements in your lattices).

'[I]f the analysis was done overnight and you only made a few changes, there shouldn't be much to do to "repair" the analysis according to your changes.'

I would be tremendously surprised if that proved to be true in the general case, but it might well be true enough in common cases to be practical. Very interesting notion...

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#118

Post by [deleted] » Fri, Jun 26, 2015, 10:12 PM UTC

seanmcdirmid wrote: post #71

danbruc wrote: post #53

Earlier quoted context omitted.

How would maintaining a SSA representation of the code help detecting dead code paths? Doing so requires reasoning about possible truth values of conditions and is in general equivalent to the halting problem and therefore undecidable. You could only spend the cycles available in real time and that will in a certain number of cases be enough to solve the problem but there will also be instances requiring more cycles…

Going incremental is another option: if the analysis was done overnight and you only made a few changes, there shouldn't be much to do to "repair" the analysis according to your changes. But the PL community is not really good with incremental computations, especially ones that must consider arbitrary code changes (which, for example, can cause non-monotonic movements in your lattices).

[deleted]

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#119

Post by paulasmuth » Sat, Jun 27, 2015, 12:32 AM UTC

jnbiche wrote: post #78

pmontra wrote: post #34

I used lint for C some 20 or 25 years ago, then moved to using dynamical languages (Perl, Ruby, JS.) Not much use of static analysis tools for those languages I thought (but for example check https://www.infinum.co/the-capsized-eight/articles/top-8-too... ). I used Java sometimes and took advantage of Netbeans/Eclipse's hints about Java code but I use emacs for everything else (some vim too). I used jslint on a JavaS…

> it's useless to define a var inside a loop No, `for` loops have nothing to do with JavaScript scope. What you probably ran into was a situation like this: for (var i = 0; i In this case, as a stylistic preference, JSLint will suggest you remove the second var, or you can also declare it at the top of the scope and then omit the var declarations in the `for` loops anyway. And yes, strictly speaking, the latter optio…

I think that was parent's point. Coming from other languages, it's kind of unexpected that a var defined within a curly brace block (if/for/etc) is actually scoped on the whole function in which it appears and not just that block. Hence it's "useless" to define it within the block/loop -- might as well hoist it outside, it's visible there anyway

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#120

Post by NickNameNick » Sat, Jun 27, 2015, 3:32 AM UTC

I ended up writing my own for the bespoke language my company uses internally. The lack of tools like java findbugs had been annoying me.

Luckily the server side 'workflow' language, and the client side templates are all written in xml, so its trivial to parse.

The first version was painfully slow. The second version tracked the range of possible states as it did a depth first search of the AST, and then stored the requirements of each node of the tree before it moved on to the nodes next sibling or parent. That way if another part of the software later called into an equivalent subtree, I could compare the current range of possible states to the requirements of the first node in the subtree and record any mismatches, then move on. It went from taking over a minute to evaluate a fairly trivial app, to parsing the biggest apps my company has ever shipped in 1-2 seconds.

I'm sure the technique is quite common. I wish I could remember if I read about it, or applied it from a different context. In any case the exercise was more useful for learning than as a finished product.