Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Impact of Programming Languages on Code Quality

arxiv.org

Post reply on HN
Print view View original

101–110 of 138 posts

Re: On the Impact of Programming Languages on Code Quality

#101

Post by darkmarmot »

joshlemer wrote: post #92
dragonwriter wrote: post #91

Earlier quoted context omitted.

> Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. Very few of the unit tests I've written for Ruby/Python catch errors that would be caught by typical type system use in C/Java/etc. (Haskell, maybe, but...) The main thing I miss in dynamic languages vs. static is the more powerful automatic testing possible when test specs can leverage type information (e.g.…

I've only ever done property based testing in Scala, but I thought it was pretty popular in dynamic languages like clojure and erlang. What would prevent it from working in Ruby/Python?

We use property testing with Elixir (erlang VM) -- and it would translate perfectly well to Ruby or Python. I also think it's a community difference, not a technical one.

Re: On the Impact of Programming Languages on Code Quality

#102

Post by pfdietz »

treebog wrote: post #94
codazoda wrote: post #67

Earlier quoted context omitted.

I thrive on checklists and hate the ad-hoc way I do code reviews on my team. This sounds great, I'm going to go look for some basic formal checklists. I need language agnostic versions and/or versions for javascript, php, and python, if anyone has suggestions.

The Art of Software Testing by Glenford Myers has an excellent language-agnostic checklist. See the chapter on “walkthroughs”

That is a book that, even in the third edition, has a paragraph that disparages random testing.

It's difficult for me to see how, in the era of jsfunfuzz, Csmith, AFL, and other such tools, such a source should be taken seriously.

Re: On the Impact of Programming Languages on Code Quality

#103

Post by munchbunny »

pfdietz wrote: post #102
treebog wrote: post #94

Earlier quoted context omitted.

The Art of Software Testing by Glenford Myers has an excellent language-agnostic checklist. See the chapter on “walkthroughs”

That is a book that, even in the third edition, has a paragraph that disparages random testing. It's difficult for me to see how, in the era of jsfunfuzz, Csmith, AFL, and other such tools, such a source should be taken seriously.

A lot of people were right in worthwhile ways and wrong in other ways. For example, Newton.

You would be doing yourself a disservice to throw out all of Newton's results because he didn't grasp relativity.

Re: On the Impact of Programming Languages on Code Quality

#104

Post by treebog »

pfdietz wrote: post #102
treebog wrote: post #94

Earlier quoted context omitted.

The Art of Software Testing by Glenford Myers has an excellent language-agnostic checklist. See the chapter on “walkthroughs”

That is a book that, even in the third edition, has a paragraph that disparages random testing. It's difficult for me to see how, in the era of jsfunfuzz, Csmith, AFL, and other such tools, such a source should be taken seriously.

I’ve never found a book on software I agreed 100% with. Even my favorites have some bits I think are dead wrong. It doesn’t mean the other parts aren’t valuable.

Re: On the Impact of Programming Languages on Code Quality

#105

Post by jrumbut »

TheOtherHobbes wrote: post #29
narag wrote: post #24

Earlier quoted context omitted.

So the old adage stands: you can write crap in any language.

And if you do, chances are good it'll end up on GitHub. 5.3 and 5.4 really nail it, IMO. And you can't even use bug fixes as a metric, because there's a difference between semantic bugs which will be significantly language-influenced, architectural bugs which are partly language-influenced, and user-expectation/experience bugs, which are broad-brush logical errors or misunderstandings. (There are probably other categ…

I'll agree with you for sure no objective metric for quality (that is broad enough to capture a satisfying amount of what we mean by quality).

I do think user experience/expectation bugs are very language dependent though. First you have the quality of the String and Datetime libraries. Ruby has all kinds of high quality string functions that are a source of pain and bugs in C.

Say your project wants to display the different between two datetimes as "5 minutes ago" or "last month" (in several languages). Saying "1.27 days ago" is correct to an extent but also not pretty. There exist high quality libraries for this, but not in every language, and a lot of devs would stumble on this feature.

The other is UI creation, I would say it is easier and faster to create a really nice UI with HTML/CSS/JS than with native Android. Maybe I'm wrong on this example but I bet there is a pair of UI frameworks where this is true.

Re: On the Impact of Programming Languages on Code Quality

#106

Post by ramchip »

dragonwriter wrote: post #93
joshlemer wrote: post #92

Earlier quoted context omitted.

I've only ever done property based testing in Scala, but I thought it was pretty popular in dynamic languages like clojure and erlang. What would prevent it from working in Ruby/Python?

Without type information already in the code (for parameters, etc.) there would be a lot more overhead in setting it up; it's possible, but higher marginal friction in the ideal case. Plus, AFAIK, no one has written the libraries, making it even higher marginal friction in practice even more than than is true in the ideal case.

Type information generally isn’t used when doing property testing in Elixir, and Python has a very mature library too (hypothesis). If there’s a difference in popularity, it could just be culture. The Erlang/Elixir community is rather fanatical about reliability.

Re: On the Impact of Programming Languages on Code Quality

#107

Post by jrumbut »

pron wrote: post #98
scott_s wrote: post #97

Earlier quoted context omitted.

I disagree that the combination of the two studies supports an effect of language choice on bug rate. Given that the original study showed a negative association for about half the languages; the replication has significant disagreement on about half of the original positive associations; and the extremely small associations across all languages, I think we're looking at noise.

But if we consider as the main result that the total effect of language choice is less than 1%, even "looking at noise" means looking at an effect of 0, and either way, the effect is tiny to nonexistent. So if someone had the hypothesis that the effect of language choice is large , because lack of evidence of a big effect is evidence of lack of one, this study is some evidence against such a hypothesis. That this fin…

If I were sitting on a thing that could decrease bugs 1% I'd be really excited. The only reason this looks like a paltry reduction is that it's hugely expensive and difficult to change languages midstream. This is evidence against a shop retraining C++ devs in Haskell (which I'm sure someone somewhere wants to do).

But if it was something like switching editor fonts? I'd do it for 1%.

Re: On the Impact of Programming Languages on Code Quality

#108

Post by voidhorse »

scott_s wrote: post #21
voidhorse wrote: post #3

Interesting! Excepting typescript, the “safer” languages also happen to be the ones I gravitate towards on a daily basis and actually enjoy writing programs in, while the others... It’d be interesting to see whether programmers’ preferences uphold the division posited by the study, or whether they are widely variable—eg is it likely that if you prefer ruby you’ll also prefer typescript over JavaScript, if you prefer…

I think you're confusing the conclusions from the original paper with the conclusions from this paper. This paper is a reproduction of the original study, and for the most part, it is not able to reproduce the findings.

How right you are. That’s what I get for skimming news piecemeal way too quickly in the mornings!

Re: On the Impact of Programming Languages on Code Quality

#109

Post by mark-r »

TheOtherHobbes wrote: post #29
narag wrote: post #24

Earlier quoted context omitted.

So the old adage stands: you can write crap in any language.

And if you do, chances are good it'll end up on GitHub. 5.3 and 5.4 really nail it, IMO. And you can't even use bug fixes as a metric, because there's a difference between semantic bugs which will be significantly language-influenced, architectural bugs which are partly language-influenced, and user-expectation/experience bugs, which are broad-brush logical errors or misunderstandings. (There are probably other categ…

> user-expectation/experience bugs

I consider those to be bugs in the spec, not the code. I once worked at a place that simply swept those under the rug - if it worked as documented, it didn't matter how many users couldn't get it to work. The bug report was closed as "not a bug"; you can imagine how frustrating that was.

Re: On the Impact of Programming Languages on Code Quality

#110

Post by yogthos »

Skunkleton wrote: post #82
yogthos wrote: post #80

It looks like the key takeaways are: Overall language effects do not appear to be a dominant factor in software quality. Manual memory management is error prone. Functional languages appear to produce better results than OO/imperaive ones. Static typing does not appear to play a measurable role. In fact, Clojure and Erlang were in a category of their own, beating out statically typed counterparts.

> Clojure and Erlang were in a category of their own I wonder how they controlled for developer skill?

It's likely that Clojure and Erlang communities have a higher ratio of experienced developers than mainstream languages. However, this would be true for Scala and Haskell communities as well presumably. Meanwhile, the fact that experienced developers choosing these languages appear to produce better code is an interesting data point of itself.
Post reply on HN