Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bug Prediction at Google

google-engtools.blogspot.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Bug Prediction at Google

#61

Post by T_S_ »

Lewisham wrote: post #55
T_S_ wrote: post #5

I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know…

Hi, I'm Chris, author of the blog/work. While I love me some type systems, I worry you're thinking a bit too code-centric. For example, let's say I describe to you a really hard problem, and you come up with a solution (that type checks!), and then we launch to users and they find a use case we never thought of, and our code suddenly breaks. I am sure this has happened to you. The problem isn't at the code level (tha…

Apologies for the perception of trollishness in my comment. The intent was merely to reflect my own personal quest to write better code. Obviously data mining has its place in a shop that produces mountains of code like Google. Not to mention other techniques like unit testing, etc.

Re: Bug Prediction at Google

#62

Post by JabavuAdams »

T_S_ wrote: post #5

I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know…

These are trivial kinds of errors. The big fuckups are invariably more subtle and people / requirements oriented.

When all you've got is a hammer ....

Re: Bug Prediction at Google

#64

Post by esrauch »

Lewisham wrote: post #46
maaku wrote: post #42

Earlier quoted context omitted.

Who said anything about refactoring? Chop it up into 10 files verbatim. If that is indeed all it takes to cheat the algorithm, it's not very useful.

I'm Chris, author of the post/work. Cheating the algorithm is indeed a possibility and is actually very easy: 1. Just don't file bug tickets. 2. If you do, don't attach the tickets to changes. 3. If you attach the tickets to changes, don't put the "bug" type on the ticket. Doing things to deliberately cheat the algorithm is likely not going to pass code review. One common misconception is that the algorithm is a stic…

I don't think the issue with the algorithm is people deliberately trying to cheat it. The problem is that files that are actually by any empirical measure more bug-free getting higher scores.

You could have two very similar pieces of code, one of which is in one larger file with 1000 lines. Imagine this code gets 1 bug per day.

Now imagine the other code that is functionally identical happens to be broken up into 10 files. Now imagine that 9 bugs per day are applied to this unit of code that is essentially identical to the 1000 line file; the broken up code is exactly the same and is 9x buggier, but it will be scored lower because each file is only getting 0.9 bugs on average per day.

Do you disagree that the code in the second example would actually be way "trickier" and deserves to have a flag on it much more than the first one? I understand that one requirement for this is clarity of the algorithm to developers, but it seems like you could easily take the ratio of bug fixes to total commits on a file or normalize by file length if you wanted to actually get a reasonable metric of how "tricky" or dangerous it is to edit a file.

Re: Bug Prediction at Google

#65

Post by esrauch »

chii wrote: post #39
esrauch wrote: post #30

Earlier quoted context omitted.

That could be true, my point was that even if a 1000 line file has a 1% chance of bug-per-line it will be marked "buggy" while a 50 line file with a 10% chance of bug-per-line won't. It makes no sense that if you took that 1000 line file and refactored it into 10 files and those 10 files had the same bugs as the original that the 1000 line file will be flagged as buggy but all 10 of the split files won't.

what you haven't thought about is that when you do refactor the 1000 line file into 10 files, that refactor would necessarily have made each bit slightly simpler (by virtue of it being smaller). Hence, it is likely that bugs don't get introduced as easily, simply because there is less to 'worry' about when writing code that has a smaller cognative load. so may be the metric isn't as wrong as you initially thought.

No, you can literally just take 10 javascript functions and put them in 10 different files or one file with absolutely no modification at all. Why would one get flagged and the other one not?

The point is simply that if you have a unit of code that has X bugs per day this will give a lower score if code that is actually buggier is across multiple shorter files.

Re: Bug Prediction at Google

#66

Post by [deleted] »

esrauch wrote: post #65
chii wrote: post #39

Earlier quoted context omitted.

what you haven't thought about is that when you do refactor the 1000 line file into 10 files, that refactor would necessarily have made each bit slightly simpler (by virtue of it being smaller). Hence, it is likely that bugs don't get introduced as easily, simply because there is less to 'worry' about when writing code that has a smaller cognative load. so may be the metric isn't as wrong as you initially thought.

No, you can literally just take 10 javascript functions and put them in 10 different files or one file with absolutely no modification at all. Why would one get flagged and the other one not? The point is simply that if you have a unit of code that has X bugs per day this will give a lower score if code that is actually buggier is across multiple shorter files.

[deleted]

Re: Bug Prediction at Google

#70

Post by jdpage »

MaysonL wrote: post #37
jdpage wrote: post #35

Earlier quoted context omitted.

The text column is narrow because it's easier for humans to read text in narrow columns. Also, negative space isn't necessarily bad -- it can be used to draw attention to important items. Now that I think about it, if the black bar was lighter, then the page wouldn't balance visually. To be honest, I didn't actually notice what was on the black bar when I first read the article, and when you scroll down it collapses…

I have no problem with the narrow text column, what bugs me is the fact that the web page has horizontal scroll bars on my 1440-pixel wide display. Nothing on the page requires that much width.

You're right, it does. Didn't notice that. That is rather silly.
Post reply on HN