Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Impact of Programming Languages on Code Quality

arxiv.org

Post reply on HN
Print view View original

21–30 of 138 posts

Re: On the Impact of Programming Languages on Code Quality

#21

Post by scott_s »

voidhorse wrote: post #3

Interesting! Excepting typescript, the “safer” languages also happen to be the ones I gravitate towards on a daily basis and actually enjoy writing programs in, while the others... It’d be interesting to see whether programmers’ preferences uphold the division posited by the study, or whether they are widely variable—eg is it likely that if you prefer ruby you’ll also prefer typescript over JavaScript, if you prefer…

I think you're confusing the conclusions from the original paper with the conclusions from this paper. This paper is a reproduction of the original study, and for the most part, it is not able to reproduce the findings.

Re: On the Impact of Programming Languages on Code Quality

#22

Post by chriswarbo »

coldcode wrote: post #19

Testing Objective-C is sort of pointless, they should have picked Swift.

Apple didn't invent Objective-C, it just-so-happened to be the language of choice at Next, which carried on into OSX and iOS. The fact that Swift is now Apple's language of choice doesn't make analysis of Objective-C "sort of pointless", any more than e.g. Facebook creating Hack would make anything involving PHP "sort of pointless".

Re: On the Impact of Programming Languages on Code Quality

#23

Post by scott_s »

A common thread I'm seeing in the discussion here is confusing the claims from the original paper with conclusions made by this paper. This is a replication study, and it starts by stating what the original paper found. What the original paper found is not what this paper finds. For the most part, this is a replication failure, for a myriad of reasons, including several kinds of misclassification.

From this paper:

The reanalysis failed to validate most of the claims of [18]. As Table 6(d-f) shows, the multiple steps of data cleaning and improved statistical modeling have invalidated the significance of 7 out of 11 languages. Even when the associations are statistically significant, their practical significance is small.

The reference [18] is https://dl.acm.org/citation.cfm?doid=2635868.2635922, which is the original paper that this paper is a replication study of.

Re: On the Impact of Programming Languages on Code Quality

#24

Post by narag »

scott_s wrote: post #23

A common thread I'm seeing in the discussion here is confusing the claims from the original paper with conclusions made by this paper . This is a replication study, and it starts by stating what the original paper found. What the original paper found is not what this paper finds. For the most part, this is a replication failure, for a myriad of reasons, including several kinds of misclassification. From this paper: T…

So the old adage stands: you can write crap in any language.

Re: On the Impact of Programming Languages on Code Quality

#25

Post by pmiller2 »

coldcode wrote: post #19

Testing Objective-C is sort of pointless, they should have picked Swift.

I disagree. If anything, they should have tested Obj-C and Swift. I would also have liked to see Scheme, Common Lisp, or something else lisp-y besides Clojure. Kotlin would have been interesting. R or Mathematica would have been fun, too.

I guess it just comes down to needing a sufficiently large corpus of code and commits and having to eventually publish.

Re: On the Impact of Programming Languages on Code Quality

#26

Post by pmiller2 »

scott_s wrote: post #23

A common thread I'm seeing in the discussion here is confusing the claims from the original paper with conclusions made by this paper . This is a replication study, and it starts by stating what the original paper found. What the original paper found is not what this paper finds. For the most part, this is a replication failure, for a myriad of reasons, including several kinds of misclassification. From this paper: T…

Yes, definitely. This is mostly a non-replication of the original study. I was a little disappointed that they didn't look at defects per line of code, even though that's already a fairly well-studied metric.

I think the lesson here is that you can write good or bad code in any language, but choose one that's fit for purpose. Using proper abstractions (including those builtin to languages) probably does more to reduce the incidence of bugs than anything you can do other than writing automated tests.

Re: On the Impact of Programming Languages on Code Quality

#27

Post by noname120 »

salmonellaeater wrote: post #10

Of note is that these are all associations. It's possible the associations are all due to selection effects (or even that the "better" language are actually worse but their effects are reversed by selecting for programmers who are better). In the early days of Python, some employers used it as a way to filter for candidates who were ahead of the curve. I believe Paul Graham mentioned Lisp being good for attracting be…

In an infamous rant, Linus Torvalds also hints at using a language as a filter for programmers[1]:

>Quite frankly, even if the choice of C were to do nothing but keep the C++ programmers out, that in itself would be a huge reason to use C

[1] http://harmful.cat-v.org/software/c++/linus

Re: On the Impact of Programming Languages on Code Quality

#28

Post by zephyrfalcon »

salmonellaeater wrote: post #10

Of note is that these are all associations. It's possible the associations are all due to selection effects (or even that the "better" language are actually worse but their effects are reversed by selecting for programmers who are better). In the early days of Python, some employers used it as a way to filter for candidates who were ahead of the curve. I believe Paul Graham mentioned Lisp being good for attracting be…

> In the early days of Python, some employers used it as a way to filter for candidates who were ahead of the curve. I believe Paul Graham mentioned Lisp being good for attracting better programmers

Possibly "The Python Paradox" [1] and/or "Beating the Averages" [2]. (The first one, from 2004, is a bit of an odd read nowadays, since it no longer applies to Python.)

[1] http://www.paulgraham.com/pypar.html [2] http://www.paulgraham.com/avg.html

Re: On the Impact of Programming Languages on Code Quality

#29

Post by TheOtherHobbes »

narag wrote: post #24
scott_s wrote: post #23

A common thread I'm seeing in the discussion here is confusing the claims from the original paper with conclusions made by this paper . This is a replication study, and it starts by stating what the original paper found. What the original paper found is not what this paper finds. For the most part, this is a replication failure, for a myriad of reasons, including several kinds of misclassification. From this paper: T…

So the old adage stands: you can write crap in any language.

And if you do, chances are good it'll end up on GitHub.

5.3 and 5.4 really nail it, IMO.

And you can't even use bug fixes as a metric, because there's a difference between semantic bugs which will be significantly language-influenced, architectural bugs which are partly language-influenced, and user-expectation/experience bugs, which are broad-brush logical errors or misunderstandings.

(There are probably other categories, but those are the obvious ones I can think of right now.)

The real takeaway is there is as yet no objective metric for software quality.

The best a working dev shop can do is implement a small but non-trivial sample project with known-good logic in a handful of candidate languages, and see what kind of trade-offs fall out.

Re: On the Impact of Programming Languages on Code Quality

#30

Post by [deleted] »

pmiller2 wrote: post #25
coldcode wrote: post #19

Testing Objective-C is sort of pointless, they should have picked Swift.

I disagree. If anything, they should have tested Obj-C and Swift. I would also have liked to see Scheme, Common Lisp, or something else lisp-y besides Clojure. Kotlin would have been interesting. R or Mathematica would have been fun, too. I guess it just comes down to needing a sufficiently large corpus of code and commits and having to eventually publish.

[deleted]
Post reply on HN