Prevention is better than cure...
So the next step would be to warn the developer before they make the code changes or during. Hooking into the IDE to visually indicate hot spots within the source file being edited on would be neat.
Bug Prediction at Google
51–60 of 71 posts
Re: Bug Prediction at Google
#52Trouble is, lots of nontrivial bugs are not module (or file) specific. The occur due to interaction between modules. Implicit assumptions on input are violated, invalid output is produced and continues propagating in the system. [Hope this technology will help make Chrome more stable. Do they even consider resource leaks bugs?]
That doesn't matter so much, because this project is about highlighting delicate code regardless of reason, not assigning blame.
Re: Bug Prediction at Google
#53Earlier quoted context omitted.
I am not surprised that you are being downvoted. In my experience, one may question the superiority of dynamic typing at HN only to the detriment of ones karma. Well, its not HN alone, dynamic languages are certainly popular and quite persuasively championed. Now I belong to the "had been persuaded before, but now I am not so sure" category. More so after discovering absolutely stupid typos and compile time checkable…
I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma. Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.
Build an HTML generator of function PreHtml -> HTML that is quirk compatible with all browsers. Yes, you will has many bugs to fix over time.
Now have all your templates code emit output of type
PreHtml and your final page generation phase is the function above.
Pay attention here: any template that attempts to emit HTML will fail to Typecheck.
Not a panacea, but static enforcement of Don't Repeat Your Bugs, even before those particular bugs are discovered.
Re: Bug Prediction at Google
#54What about files with limited or no history. Cycolmatic Complexity [1] to the rescue?
Re: Bug Prediction at Google
#55I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know…
Hi, I'm Chris, author of the blog/work.
While I love me some type systems, I worry you're thinking a bit too code-centric.
For example, let's say I describe to you a really hard problem, and you come up with a solution (that type checks!), and then we launch to users and they find a use case we never thought of, and our code suddenly breaks. I am sure this has happened to you. The problem isn't at the code level (that's just the manifestation), but our current understanding of the problem.
IMHO, once you reach a certain level of competency, a sizable minority, if not majority, of bugs that make their way into a source control system (particularly one with code review, like Google's) are not line-by-line problems, but wider problems with actually understanding the task at hand and all that encompasses. When you work on the complexity of problems that Google does, you're tending towards a very large chunk of bugs being members of this class.
However, we don't have good tools (yet) to have a computer properly check the semantic meaning of our code. Bug prediction sits as a sort of baby step. It's the computer making a best-effort guess of where issues will be. The algorithm designed here is saying "Look this file has been a struggle. It's probably going to continue being a struggle, so we need to focus attention when we change anything to it."
Re: Bug Prediction at Google
#56Prevention is better than cure... So the next step would be to warn the developer before they make the code changes or during. Hooking into the IDE to visually indicate hot spots within the source file being edited on would be neat.
Yes, this is exactly right. We didn't have time in the period of an internship to hook into the IDE (along with all the other tasks, like user studies, algorithm design etc etc), so just went with the warning in source control.
Given more time, I absolutely would have done this. Nicolas Lopez at UC Irvine is thinking about putting tools like this at the IDE level, if you're interested.
Re: Bug Prediction at Google
#57I'll recommend Nicholas Taleb's "The Black Swan" to the authors of this prediction tool. Statistics cannot be used to "predict" future behavior of a complex system.
Re: Bug Prediction at Google
#58I'll recommend Nicholas Taleb's "The Black Swan" to the authors of this prediction tool. Statistics cannot be used to "predict" future behavior of a complex system.
Statistics cannot be used to "predict" future behavior of a complex system
They certainly can, and commonly are! Think of weather prediction, Bayesian diagnostic systems, Google's statistical translator and spell checker, etc etc.
Taleb's point is that they fail sometimes - perhaps more often than you think - and so you should be very careful in placing bets on statistical predictions.
In this case the cost of a failed prediction is very low, and a successful prediction is worth quite a lot, so it is a net win for Google.
Re: Bug Prediction at Google
#59https://github.com/igrigorik/bugspots - for anyone that's curious to try it on their own git repo. (improvements welcome! :-))
looks good, but you might move the pattern matching in scanner.rb out to a config file so users can search for other keywords.
Re: Bug Prediction at Google
#60I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know…
Hi, I'm Chris, author of the blog/work. While I love me some type systems, I worry you're thinking a bit too code-centric. For example, let's say I describe to you a really hard problem, and you come up with a solution (that type checks!), and then we launch to users and they find a use case we never thought of, and our code suddenly breaks. I am sure this has happened to you. The problem isn't at the code level (tha…
[deleted]