Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

viva64.com

Post reply on HN
Print view View original

31–40 of 130 posts

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#31

Post by raz32dust »

dschiptsov wrote: post #17

Because neither an advanced type system nor static analysis could catch bugs in program logic?) The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

> The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

I have no stats here (and neither do you :)), but based on my experience, Java code does tend to be much less buggy when compared with dynamic-typed code, keeping the features and quality of developers the same. Of course, logical bugs don't get caught by static typing. But it helps a lot when refactoring code, or collaborating on the same codebase with many people, or changing someone else's code. These things become really important once the code base hits a certain size.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#32

Post by glogla »

amirmc wrote: post #23
dschiptsov wrote: post #17

Because neither an advanced type system nor static analysis could catch bugs in program logic?) The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

This reasoning makes no sense to me. It's like: Alice: "Doing X will prevent bugs like Y!" Bob: "Oh, but it does nothing for bugs like Z. I just won't bother at all, then". Why would you not want to try and remove an entire class of bugs if it were within your power to do so? Just look at all the effort companies like Facebook have poured into exactly this kind of problem with things like Hack and Flow (which make us…

I think you're missing each other in terms of what static typing you mean.

Haskell or ML style static typing is very useful, completely changes the way you do things and gives you many guarantees.

C or Java static typing is almost useless with regards to bugs (see null) and serves mostly to annoy you. It has many of extra costs of stronger static typing, but gives you very little of the benefits.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#33

Post by geromek »

PinguTS wrote: post #19
geromek wrote: post #14

I think the reason is fairly simple: even the most evident-no-doubt clear SQL Injection vulnerability found by a SCA tool may never be exploited at all under production (for instance because of a WAF). Then the obvious benefits of static analysis are not that obvious for your employer. Sometimes we forget companies do not want a perfect code or the best possible well designed software but a product that make them ear…

So, you are saying "Developers don't want to deliver quality."? If that it true, than I don't want to work with them.

I am saying "Developers do not want : 1 - Pay A LOT of money for advanced solutions that are more than AST checkers (hello SonarQube) or big piles of false positives. 2 - Add overhead to their workflows (more than an IDE plugin is harmful, and what happens with those devs not using an IDE?). 3 - Spend time on figuring out if the static analysis results make sense or not, one by one.

A typical SCA tool can report hundreds or thousands of occurrences for a certain code base. How are developers going to deal with them?

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#34

Post by pmontra »

I used lint for C some 20 or 25 years ago, then moved to using dynamical languages (Perl, Ruby, JS.) Not much use of static analysis tools for those languages I thought (but for example check https://www.infinum.co/the-capsized-eight/articles/top-8-too...). I used Java sometimes and took advantage of Netbeans/Eclipse's hints about Java code but I use emacs for everything else (some vim too).

I used jslint on a JavaScript project a few months ago. It catched a few things I overlooked, I learned a couple of things (example: it's useless to define a var inside a loop) but it's also very opinionated and insisted that I write code in a way that pleases its author. I had to refactor working code to make most of the warnings go away. I ended up with something that worked as well as the original but was much more complicated to read (at least for me). I stopped using it and regretted making all those changes to the code. There are some configuration switches but nothing that could make it work for me. I should check jshint but unfortunately jslint primed me against that kind of tools.

I prefer to catch errors with tests and test coverage tools. They must be used anyway and they bend to me, not the other way around.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#35

Post by taeric »

glogla wrote: post #32
amirmc wrote: post #23

Earlier quoted context omitted.

This reasoning makes no sense to me. It's like: Alice: "Doing X will prevent bugs like Y!" Bob: "Oh, but it does nothing for bugs like Z. I just won't bother at all, then". Why would you not want to try and remove an entire class of bugs if it were within your power to do so? Just look at all the effort companies like Facebook have poured into exactly this kind of problem with things like Hack and Flow (which make us…

I think you're missing each other in terms of what static typing you mean. Haskell or ML style static typing is very useful, completely changes the way you do things and gives you many guarantees. C or Java static typing is almost useless with regards to bugs (see null) and serves mostly to annoy you. It has many of extra costs of stronger static typing, but gives you very little of the benefits.

Static analysis is more than just static typing. Notably, it is possible to leverage static analysis for solid gains in reliability against code bases without having to rewrite them to take advantage of a different typing paradigm.

To repeat, you can harden a code base without rewriting it by using static analysis tools. This is not as true with static typing.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#36

Post by taeric »

pmontra wrote: post #34

I used lint for C some 20 or 25 years ago, then moved to using dynamical languages (Perl, Ruby, JS.) Not much use of static analysis tools for those languages I thought (but for example check https://www.infinum.co/the-capsized-eight/articles/top-8-too... ). I used Java sometimes and took advantage of Netbeans/Eclipse's hints about Java code but I use emacs for everything else (some vim too). I used jslint on a JavaS…

It is the "opinions enforced with static analysis" that ultimately annoy me with most analysis tools out there. Things that actually cause bugs, I welcome ways of finding them. Chaining all of my variable declarations together... What?

The sad thing is that the powerful analysis tools are truly amazing and do find things. They just are not what the majority of people have experience with. That does seem to be changing.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#37

Post by flohofwoe »

dschiptsov wrote: post #17

Because neither an advanced type system nor static analysis could catch bugs in program logic?) The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

Have you looked at the clang analyzer that's integrated into Xcode (http://clang-analyzer.llvm.org/)? It doesn't catch everything, and sometimes gives false positives, but it is quite amazing how it visualizes problems in the program flow, going through several levels in the call hierarchy, following variable assignments over function calls, etc... in a few cases I was convinced that a problem was a false positive until I really took the time to understand what the static analyzer is trying to tell me and it turned out to be actual bugs.

In my opinion it is mostly poor integration into the coding workflow, and poor visualization of the analyzer warnings that lead to too little use of static code analysis, and another is a too wide-spread 'if it works at all, ship it' mentality.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#38

Post by PinguTS »

geromek wrote: post #33
PinguTS wrote: post #19

Earlier quoted context omitted.

So, you are saying "Developers don't want to deliver quality."? If that it true, than I don't want to work with them.

I am saying "Developers do not want : 1 - Pay A LOT of money for advanced solutions that are more than AST checkers (hello SonarQube) or big piles of false positives. 2 - Add overhead to their workflows (more than an IDE plugin is harmful, and what happens with those devs not using an IDE?). 3 - Spend time on figuring out if the static analysis results make sense or not, one by one. A typical SCA tool can report hund…

I am from engineering background and not soley a software guy, so forgive me my different view on this topic.

I learned, that every error you can fix early on will cost you about 10x to fix in the next stage.

All the new principles like Agile have not changed that.

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#39

Post by roel_v »

72deluxe wrote: post #25

I do! For cheapness, Flawfinder, CPPCheck, and Product > Analyze in xcode (thanks clang) (the walkthroughs in xcode of how they deduced the problem are really great, blue arrows everywhere, very neat). EDIT: Additionally, given the changes in C++11 we should be able to push all the work onto the compiler for checking type problems, remove dangerous operations, dangling pointers, naked new/delete and avoid casting whe…

I refactored some code this week that used realloc to allocate memory for a 'vector' (which was a typedef for float*), and then a loop with additional realloc's to allocate memory for each element in that vector, to end up with a matrix 8| This is code that was written in the last few years, too. Welcome to academia :(

Re: Why Don't Software Developers Use Static Analysis Tools to Find Bugs?

#40

Post by Xixi »

dschiptsov wrote: post #17

Because neither an advanced type system nor static analysis could catch bugs in program logic?) The benefits of static typing (complie-time checks) are grossly exaggerated. If the claims were true, Java itself and Java projects would be much less buggy.)

I would not say grossly exaggerated, especially for languages with stronger type systems than Java. But the worst bugs I've encountered definitely came from code behaving exactly as intended. The flawless implementations of terrible, terrible ideas.

There's not much a language can do to protect a programmer from doing nonsense. Terse, explicit languages that are easy to reason about have a slight edge in that they make it easier to grasp the big picture. A powerful type system like that of Haskell can certainly help too. Practices that involves many eyes looking at the code help the most.

Post reply on HN