On the Impact of Programming Languages on Code Quality • Hacker News Board

Re: On the Impact of Programming Languages on Code Quality

#81

Post by gnaritas » Wed, Jun 05, 2019, 7:58 PM UTC

stcredzero wrote: post #52

robbrit wrote: post #46

Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of these findings: * Python, JS, PHP, Java, C++ are all very common first languages; I'd imagine the average experience level for these is dragged down a bit compared to others like Golang, Haskell, or Scala. One of the recent Stack Overflow Developer surveys found G…

Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. So then, could the argument be made that less stringent type annotation hurts code quality by removing effort from unit testing?

No, the behavioral test [1] one writes cannot be replaced by type checks nor does a type system reduce the need to write tests if you're doing testing correctly.

[1] https://twitter.com/unclebobmartin/status/113589437016571084...

Re: On the Impact of Programming Languages on Code Quality

#82

Post by Skunkleton » Wed, Jun 05, 2019, 7:59 PM UTC

yogthos wrote: post #80

It looks like the key takeaways are: Overall language effects do not appear to be a dominant factor in software quality. Manual memory management is error prone. Functional languages appear to produce better results than OO/imperaive ones. Static typing does not appear to play a measurable role. In fact, Clojure and Erlang were in a category of their own, beating out statically typed counterparts.

> Clojure and Erlang were in a category of their own

I wonder how they controlled for developer skill?

Re: On the Impact of Programming Languages on Code Quality

#83

Post by Zak » Wed, Jun 05, 2019, 8:04 PM UTC

stcredzero wrote: post #52

robbrit wrote: post #46

Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of these findings: * Python, JS, PHP, Java, C++ are all very common first languages; I'd imagine the average experience level for these is dragged down a bit compared to others like Golang, Haskell, or Scala. One of the recent Stack Overflow Developer surveys found G…

Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. So then, could the argument be made that less stringent type annotation hurts code quality by removing effort from unit testing?

Python is a language fundamentally similar to Ruby, that has no static type checking, that's used in similar application domains, etc.... It does not have as strong a culture of unit testing as Ruby, and falls above the line (by a statistically insignificant amount) in figure 5 while Ruby is below (by a statistically significant amount).

It does not appear that the paper took automated testing into account at all. Whether greater use of automated testing in Ruby is responsible for the lower defect rate observed here relative to Python would definitely be an interesting topic for another paper.

Clojure is also dynamically typed by default and below the line. It does not have Ruby's strong unit testing culture, but it does have a design that encourages functional programming, which proponents argue reduces sources of error. It also has a culture of REPL-driven development that arguably fills some of the role of automated testing.

Re: On the Impact of Programming Languages on Code Quality

#84

Post by tsegratis » Wed, Jun 05, 2019, 8:08 PM UTC

I wonder if this lack of significant link is because number_of_bugs = time_spent_programming / time_fixing_bugs

So choice of language does affect the number of potential bugs. But this ratio of effort a programmer gives to bugs vs features is almost completely independent. Actual bugs making it past the programmer depend on how much effort the programmer cares to expend

So a safer language doesn't reduce bugs. It's like farmers with back pain from driving tractors. Giving them comfier rides doesn't reduce their pain; it just means they drive faster upto the same pain limit

If this is true, then bugs can be reduced by making bugs more painful for the programmer

But also a safer language will let the programmer drive at a higher speed, releasing a greater percentage of safe code (even if the total number of bugs is the same)

Re: On the Impact of Programming Languages on Code Quality

#85

Post by ofrzeta » Wed, Jun 05, 2019, 8:12 PM UTC

baby wrote: post #35

I’ve read a lot of code as a consultant. The best codebase I’ve read had almost no comments and was written in erlang. Besides that every codebase I’ve read in Golang were super clear and of excellent quality for the exception of one that was written by java developers. I can say I hated reading C because of maccros and different build systems. I hated C++ codebases because people abused generics. And I hated Java th…

> .. the layers of abstractions and all the factories and all the singletons and all the...

You can have that in PHP, too, when the project builds on Symfony, that is to say the style is not only dictated by the language but also on the framework used.

Re: On the Impact of Programming Languages on Code Quality

#86

Post by moogly » Wed, Jun 05, 2019, 8:17 PM UTC

JamesBarney wrote: post #48

robbrit wrote: post #46

Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of these findings: * Python, JS, PHP, Java, C++ are all very common first languages; I'd imagine the average experience level for these is dragged down a bit compared to others like Golang, Haskell, or Scala. One of the recent Stack Overflow Developer surveys found G…

JS also has more ways to shoot yourself in the foot than any other language I've worked with.

Great linting tools available though. Can't say the same for many systems languages. And the static analysis tools for those tend to be quite expensive, bulky and slow.

Re: On the Impact of Programming Languages on Code Quality

#87

Post by moogly » Wed, Jun 05, 2019, 8:21 PM UTC

shados wrote: post #62

cestith wrote: post #47

Earlier quoted context omitted.

That would be interesting. I also have a feeling it would be incredibly difficult to accurately control for social differences in language communities.

Definitely. Heck, in .NET's early days, there was a distinct difference in quality of the software and of the team using VB.net vs C#, when they were 99% the same language (they had some features unique to each, but they were very few). But C# was used a lot in new projects or by people familiar with Java while VB.net was used in VB6 migrations or teams coming from VB6. That had a drastic effect on the end result, bu…

The VB shops I've seen code from tended to really like `On Error Resume Next`, too...

Sure, I've seen global try/catch blocks swallowing all errors in C#/C++ code too, but definitely not as prevalent in my anecdata.

Re: On the Impact of Programming Languages on Code Quality

#88

Post by nullc » Wed, Jun 05, 2019, 8:27 PM UTC

Good to see better statistics being applied.

However, the base methodology appears to be really weak.

The presence of fixes doesn't necessarily reflect the presence of underlying bugs: highly conscientious projects make many "fixes" that improve the software quality against an abstract ideal without directly fixing any defect, while highly unconscientious projects don't bother fixing anything.

There are also many confounders. For example, I've seen project institute restrictions on refactoring changes in response to the huge amount of brownian-code-motion popular projects can receive, only to then have some contributors start misleading describing their commits, "by splitting this function up, we avoid the risk that someone would later introduce an overflow!".

Considering how important software quality is to contemporary engineering, including life safety critical systems, it's disappointing that we don't see more randomize studies. E.g. take a pool of developers randomly assign them to teams using different tools each to accomplish the same task, then compare the results against each other and a hidden test suite.

Re: On the Impact of Programming Languages on Code Quality

#89

Post by stcredzero » Wed, Jun 05, 2019, 8:29 PM UTC

igouy wrote: post #73

stcredzero wrote: post #59

Earlier quoted context omitted.

Figuratively speaking. None of the languages they test enforce any unit testing. The trivial type-mistake "Unhandled Exceptions" in Smalltalk would leak out more often than was comfortable, if the project didn't have a good means of testing to catch those. I suspect that has something to do with the Ruby community being more obsessive about testing.

> "Unhandled Exceptions" in Smalltalk ? MessageNotUnderstood

Right. This, and many others, would result in an "Unhandled Exception" notifier dialog in many versions of Smalltalk. I used to work for the company acting as the vendor of 2 (3?) commercial Smalltalks, also doing consulting work for others.

Re: On the Impact of Programming Languages on Code Quality

#90

Post by ofrzeta » Wed, Jun 05, 2019, 8:39 PM UTC

mcguire wrote: post #78

shereadsthenews wrote: post #40

Earlier quoted context omitted.

Considering the astonishing density of defects in Linux we can treat its author’s advice on software development as anti-advice.

What is the density of defects in Linux? Any published comparisons?

According to Coverity, the "defect density" of the Linux kernel is around 0.5 errors per 1000 lines of code, while the average of the analyzed open source projects is around 0.7. Best among the analyzed projects is Python (the language/stdlib I guess) with an error rate of 0.08.