Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bug Prediction at Google

google-engtools.blogspot.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Bug Prediction at Google

#31

Post by tedunangst »

srean wrote: post #18
T_S_ wrote: post #5

I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know…

I am not surprised that you are being downvoted. In my experience, one may question the superiority of dynamic typing at HN only to the detriment of ones karma. Well, its not HN alone, dynamic languages are certainly popular and quite persuasively championed. Now I belong to the "had been persuaded before, but now I am not so sure" category. More so after discovering absolutely stupid typos and compile time checkable…

I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma.

Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.

Re: Bug Prediction at Google

#32

Post by srean »

tedunangst wrote: post #31
srean wrote: post #18

Earlier quoted context omitted.

I am not surprised that you are being downvoted. In my experience, one may question the superiority of dynamic typing at HN only to the detriment of ones karma. Well, its not HN alone, dynamic languages are certainly popular and quite persuasively championed. Now I belong to the "had been persuaded before, but now I am not so sure" category. More so after discovering absolutely stupid typos and compile time checkable…

I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma. Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.

That there exist errors that cannot be type-checked makes type-checking irrelevant ? I dont quite see why exhaustiveness is a pre-requisite for relevance or attention. Is that a thumb rule you use for other things as well ?

I am not sure if type-checking even aspires to be exhaustive. As long as it can detect my stupid and costly mistakes at a cost that is cheaper than the cost of the mistakes, I am happy. But I can understand that for people who do not make costly and compile time checkable mistakes, it might not be worth it.

@tedunangst Got you. Your second paragraph in the previous comment threw me off. It does read like you consider type-checking unworthy of attention because it cannot catch all possible errors. Agree with your comment about people overstating their case. I think the noisiest of both the camps (if one can call it that) are equally guilty.

Re: Bug Prediction at Google

#33

Post by tedunangst »

srean wrote: post #32
tedunangst wrote: post #31

Earlier quoted context omitted.

I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma. Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.

That there exist errors that cannot be type-checked makes type-checking irrelevant ? I dont quite see why exhaustiveness is a pre-requisite for relevance or attention. Is that a thumb rule you use for other things as well ? I am not sure if type-checking even aspires to be exhaustive. As long as it can detect my stupid and costly mistakes at a cost that is cheaper than the cost of the mistakes, I am happy. But I can…

If the original post does not mean to convey "I don't need this because I have ghc", well, it kinda sounds like it does. I didn't say type checking was irrelevant. I said its merits are irrelevant in a discussion about how to find bugs that have already been checked in. Unless the devs are really sloppy, such code has already been compiled and run through whatever more or less exhaustive checking that entails.

Type checking is useful. I like it. But I don't like people talking about it like it's magic pixie dust.

Re: Bug Prediction at Google

#34

Post by cpeterso »

nl wrote: post #17
esrauch wrote: post #2

Without considering the number of submits or file length you will end up with files that are simply longer and edited more getting undue high scores. A single 100 line file will be scored to be twice as buggy as the exact same code split into two 50 line files. Considering source file lengths roughly follow a Zipfian type distribution, it seems like at least the top 5% length files would get flagged regardless unless…

A single 100 line file will be scored to be twice as buggy as the exact same code split into two 50 line files. I suspect that isn't the case. Subjectively, longer files are harder to understand and in languages like Java (one file = one class, ignore inner classes etc) is likely to represent more complicated and tightly coupled code. If the same file keeps appearing, then split it in half and see what happens...

The two 50-line files will probably have better internal cohesion, making them easier to test and review than one 100-line file.

Re: Bug Prediction at Google

#35

Post by jdpage »

MaysonL wrote: post #12
tibbon wrote: post #11

Also of note, this blog is beautiful in the classic view.

I'd vehemently differ, other than perhaps as a piece of abstract art: the black bar is attention-grabbing to negative effect, and the opening animation is self-indulgent, as is the massive whitespace atop. Also, the width of the page seems rather strange: why so wide, with no content?

The text column is narrow because it's easier for humans to read text in narrow columns. Also, negative space isn't necessarily bad -- it can be used to draw attention to important items.

Now that I think about it, if the black bar was lighter, then the page wouldn't balance visually. To be honest, I didn't actually notice what was on the black bar when I first read the article, and when you scroll down it collapses into something less visually imposing. The open animation could do with being a bit faster or vanishing altogether though. I feel like having a smooth close animation is important because the eye is attracted to sudden movement.

Re: Bug Prediction at Google

#36

Post by dilap »

esrauch wrote: post #30
tesseract wrote: post #8

Earlier quoted context omitted.

I've seen a few studies (sorry, no citations handy, if I have time I'll revisit this comment) which concluded that above some fairly low floor, bugginess is strongly positively correlated with file length. Which is to say, always flagging the top 5% longest files as being among the buggiest has a good chance of being the right thing to do.

That could be true, my point was that even if a 1000 line file has a 1% chance of bug-per-line it will be marked "buggy" while a 50 line file with a 10% chance of bug-per-line won't. It makes no sense that if you took that 1000 line file and refactored it into 10 files and those 10 files had the same bugs as the original that the 1000 line file will be flagged as buggy but all 10 of the split files won't.

Not necessarily, because maybe refactoring your code to be clean enough to fit into ten files is also makes it clean enough to remove bugs. And/or hairy code doesn't end up in short files.

Re: Bug Prediction at Google

#37

Post by MaysonL »

jdpage wrote: post #35
MaysonL wrote: post #12

Earlier quoted context omitted.

I'd vehemently differ, other than perhaps as a piece of abstract art: the black bar is attention-grabbing to negative effect, and the opening animation is self-indulgent, as is the massive whitespace atop. Also, the width of the page seems rather strange: why so wide, with no content?

The text column is narrow because it's easier for humans to read text in narrow columns. Also, negative space isn't necessarily bad -- it can be used to draw attention to important items. Now that I think about it, if the black bar was lighter, then the page wouldn't balance visually. To be honest, I didn't actually notice what was on the black bar when I first read the article, and when you scroll down it collapses…

I have no problem with the narrow text column, what bugs me is the fact that the web page has horizontal scroll bars on my 1440-pixel wide display. Nothing on the page requires that much width.

Re: Bug Prediction at Google

#39

Post by chii »

esrauch wrote: post #30
tesseract wrote: post #8

Earlier quoted context omitted.

I've seen a few studies (sorry, no citations handy, if I have time I'll revisit this comment) which concluded that above some fairly low floor, bugginess is strongly positively correlated with file length. Which is to say, always flagging the top 5% longest files as being among the buggiest has a good chance of being the right thing to do.

That could be true, my point was that even if a 1000 line file has a 1% chance of bug-per-line it will be marked "buggy" while a 50 line file with a 10% chance of bug-per-line won't. It makes no sense that if you took that 1000 line file and refactored it into 10 files and those 10 files had the same bugs as the original that the 1000 line file will be flagged as buggy but all 10 of the split files won't.

what you haven't thought about is that when you do refactor the 1000 line file into 10 files, that refactor would necessarily have made each bit slightly simpler (by virtue of it being smaller). Hence, it is likely that bugs don't get introduced as easily, simply because there is less to 'worry' about when writing code that has a smaller cognative load. so may be the metric isn't as wrong as you initially thought.

Re: Bug Prediction at Google

#40

Post by chii »

nl wrote: post #23
_delirium wrote: post #16

Earlier quoted context omitted.

Your last statement's the part I disagree with, though it depends on what you mean by "orthogonal". If you mean that they're two different approaches that can coexist, then yes. But I don't think they target orthogonal classes of bugs. In both cases, the goal is to employ some algorithmic, decidable method at compile-time to predict whether a given piece of code is "correct" or "incorrect", trading off the possibilit…

>The ambition, at least, of static type systems is to render accurate bug-prediction impossible, because every bug statically predictable at compile-time will be a type error. I might not use Haskell, but I do understand what a type system does, and why it is important. BUT, not all bugs are statically predictable at compile-time . Take things like cross-browser compatibility - something like GWT goes a long way to r…

so your thesis is that a bug prediction system ought to be able to predict bugs that are not possible to statically predict?

I suppose that is a reasonable argument. i think different people in this argument are arguing about different things.

Post reply on HN