Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Impact of Programming Languages on Code Quality

arxiv.org

Post reply on HN
Print view View original

121–130 of 138 posts

Re: On the Impact of Programming Languages on Code Quality

#121

Post by pron »

yogthos wrote: post #119
bunderbunder wrote: post #115

Earlier quoted context omitted.

There's a concept I discovered in the realm of political science, the "fundamentally unanswerable question" - ones where it's not actually possible to generate the kinds of counterfactuals you'd need to try and run an experiment. I think this is probably one of those. Haskell and C++ are very different languages, that are designed for very different purposes, and are popular in very different problem domains. To star…

I really don't see how that would be the case. Both Haskell and C++ are general purpose languages, there are plenty of projects written in many overlapping domains. The test would to treat projects as black boxes, and look for trends across many projects. If you saw a trend that Haskell projects are consistently delivered faster by smaller teams, then you could make a hypothesis that Haskell is a factor. At the end o…

As someone who develops mostly in C++, I can tell you that productivity is a very small concern. We are well aware that most GCed languages are significantly more productive. Virtually the only reason people use C++ nowadays is for performance (and/or footprint) (that is partly because we have seen reports that Java delivers better productivity in the early '00s, so those who care about that have switched a long time ago; in fact, the effect was so large that they didn't have a choice). Now, if someone showed that Haskell has a significantly better performance/footprint than C++ that may cause someone to shift.

Re: On the Impact of Programming Languages on Code Quality

#122

Post by imtringued »

kazinator wrote: post #39

The problem is that "code quality" isn't just bugs per line of code. Maintainability, extensibility, readability, expressivity all factor into it. Suppose I have to write 100,000 lines in language B to write something doable in 10,000 lines of A. Who cares if the defect rate is twice as high per line of A: I still have five times fewer defects due to ten times fewer LOC's. Another thing to consider is the nature of d…

If the defect rate is the same across languages then you're not measuring the defect rate of the programming language. You're measuring the defect rate of the developers!

Re: On the Impact of Programming Languages on Code Quality

#123

Post by imtringued »

tabtab wrote: post #33
ggregoire wrote: post #32

Casual observation: adding a framework to any language seems to improve the overall code quality. It adds and teaches structure, patterns and good practices that developers usually follow. I've seen terrible codebases written in vanilla PHP, JavaScript, Python (probably the worst ones). But I've seen very good looking, easy to understand, easy to maintain codebases written in Symfony, React and Django. (React is an i…

I agree. Our framework, libraries, and/or interfaces can and should make a bigger difference than programming language. If you have to use the "weird parts" of a language to get typical work done, you are doing something wrong. And if you are not using the weird parts, then programming languages look and do pretty much the same thing. Braggings such as, "look! My language can do double recursive lambda backflips whil…

In my experience productivity is strongly correlated with the quality of frameworks and libraries that are available to a language. Java as a language might be fine but batshit insane frameworks like JSF really drag your productivity down. There are also solid libraries available but they are often too complicated and hard to use. Languages built on top of the JVM often have very very thin wrappers that increase the usability hundredfold.

Re: On the Impact of Programming Languages on Code Quality

#124

Post by neves »

mcguire wrote: post #74
neves wrote: post #42

Earlier quoted context omitted.

The practice that most reduces the incidence of tests is code review: https://kevin.burke.dev/kevin/the-best-ways-to-find-bugs-in-...

I think you mean "bugs", not tests.

Sure! thanks for the correction.

Re: On the Impact of Programming Languages on Code Quality

#125

Post by yogthos »

bunderbunder wrote: post #120
yogthos wrote: post #119

Earlier quoted context omitted.

I really don't see how that would be the case. Both Haskell and C++ are general purpose languages, there are plenty of projects written in many overlapping domains. The test would to treat projects as black boxes, and look for trends across many projects. If you saw a trend that Haskell projects are consistently delivered faster by smaller teams, then you could make a hypothesis that Haskell is a factor. At the end o…

Quantum mechanics is really a very different beast from the social sciences. Experimental physicists don't have to tease out any confounding effects due to the life history and social baggage that electrons bring with them to the lab. A huge, huge, huge part of the replication crisis in social sciences is due to researchers getting overconfident about the capabilities of their experimental methods.

Measuring quality of software products is not a social science. You have clear metrics on time it takes to develop something, number of people you need to develop it, numbers of defects per project, and so on. Once you have massive amounts of data, as we now have with repositories like GitHub, it becomes possible to look for trends across languages.

I'm not saying there aren't challenges associated with measuring these things, as there obviously are, but it's certainly possible to study them. And the more studies we have, the more confidence we get regarding the effects.

Re: On the Impact of Programming Languages on Code Quality

#126

Post by yogthos »

pron wrote: post #121
yogthos wrote: post #119

Earlier quoted context omitted.

I really don't see how that would be the case. Both Haskell and C++ are general purpose languages, there are plenty of projects written in many overlapping domains. The test would to treat projects as black boxes, and look for trends across many projects. If you saw a trend that Haskell projects are consistently delivered faster by smaller teams, then you could make a hypothesis that Haskell is a factor. At the end o…

As someone who develops mostly in C++, I can tell you that productivity is a very small concern. We are well aware that most GCed languages are significantly more productive. Virtually the only reason people use C++ nowadays is for performance (and/or footprint) (that is partly because we have seen reports that Java delivers better productivity in the early '00s, so those who care about that have switched a long time…

Sure, in some domains you have to use specific languages because of the constraints of the domain. However, as you yourself state, a language can play a significant impact when it comes to productivity.

For example, immutability provides similar benefits to GC. Instead of the developer having to manually manage references across the project by hand, the language takes care of that work. This frees the developer from doing additional work, and removes a source of errors. This also directly leads to the ability to do local reasoning about code resulting in developers needing less context to understand code. So, if GC plays a significant enough role then it's highly likely that immutability does as well because it addresses a similar set of problems.

Re: On the Impact of Programming Languages on Code Quality

#127

Post by bunderbunder »

yogthos wrote: post #125
bunderbunder wrote: post #120

Earlier quoted context omitted.

Quantum mechanics is really a very different beast from the social sciences. Experimental physicists don't have to tease out any confounding effects due to the life history and social baggage that electrons bring with them to the lab. A huge, huge, huge part of the replication crisis in social sciences is due to researchers getting overconfident about the capabilities of their experimental methods.

Measuring quality of software products is not a social science. You have clear metrics on time it takes to develop something, number of people you need to develop it, numbers of defects per project, and so on. Once you have massive amounts of data, as we now have with repositories like GitHub, it becomes possible to look for trends across languages. I'm not saying there aren't challenges associated with measuring the…

But you're not just measuring the quality of software products in isolation. The whole point is to measure how a certain set of conditions, particularly the choice of an artifically constructed language that humans use, among other things, to communicate with each other, affects the result of a human enterprise that's typically performed socially.

For these kinds of research questions, human social behavior is not a distraction from the issue being studied. It's the very locus of the issue being studied.

Re: On the Impact of Programming Languages on Code Quality

#128

Post by yogthos »

bunderbunder wrote: post #127
yogthos wrote: post #125

Earlier quoted context omitted.

Measuring quality of software products is not a social science. You have clear metrics on time it takes to develop something, number of people you need to develop it, numbers of defects per project, and so on. Once you have massive amounts of data, as we now have with repositories like GitHub, it becomes possible to look for trends across languages. I'm not saying there aren't challenges associated with measuring the…

But you're not just measuring the quality of software products in isolation. The whole point is to measure how a certain set of conditions, particularly the choice of an artifically constructed language that humans use, among other things, to communicate with each other, affects the result of a human enterprise that's typically performed socially. For these kinds of research questions, human social behavior is not a…

First, we need to established whether there is something that needs to be studied or not. The idea is to look for the outliers by exemaning a large volume of projects in many different languages. If you see a correlation between a language and a result then you can mke a hypothesis that the language is a factor.

At that point you can make a further hypothesis as to whether certain languages encourage a particular kind of social behavior, or whether particular kinds of people are drawn to particular languages. And of course it could just be the technical features of the language that end up playing a role because social aspect may be consistent across languages.

Re: On the Impact of Programming Languages on Code Quality

#129

Post by dragonwriter »

ramchip wrote: post #106
dragonwriter wrote: post #93

Earlier quoted context omitted.

Without type information already in the code (for parameters, etc.) there would be a lot more overhead in setting it up; it's possible, but higher marginal friction in the ideal case. Plus, AFAIK, no one has written the libraries, making it even higher marginal friction in practice even more than than is true in the ideal case.

Type information generally isn’t used when doing property testing in Elixir, and Python has a very mature library too (hypothesis). If there’s a difference in popularity, it could just be culture. The Erlang/Elixir community is rather fanatical about reliability.

Hypothesis uses the same type annotations as Python's optional typechecker as a tool to select appropriate input data generators without additional test-specific description. That's a fairly common feature of property-based testing tools; I don't know for sure, but I wouldn't be surprised to find that hypothesis and mypy tend to be used in the same segments of the Python community.

Re: On the Impact of Programming Languages on Code Quality

#130

Post by the_why_of_y »

yogthos wrote: post #125
bunderbunder wrote: post #120

Earlier quoted context omitted.

Quantum mechanics is really a very different beast from the social sciences. Experimental physicists don't have to tease out any confounding effects due to the life history and social baggage that electrons bring with them to the lab. A huge, huge, huge part of the replication crisis in social sciences is due to researchers getting overconfident about the capabilities of their experimental methods.

Measuring quality of software products is not a social science. You have clear metrics on time it takes to develop something, number of people you need to develop it, numbers of defects per project, and so on. Once you have massive amounts of data, as we now have with repositories like GitHub, it becomes possible to look for trends across languages. I'm not saying there aren't challenges associated with measuring the…

> numbers of defects per project

How do you measure that?

You could look at the project's bug tracker, but the number of bugs in it will reflect a project's popularity more than anything else.

You could look at commit messages, as the (original) paper does (see 2.2.1 / 2.2.2 in the reproduction paper), but the heuristic they use doesn't look very reliable to me; I've seen projects where every feature implementation had to have an issue in the bug tracker and every commit implementing that feature had to reference the issue, so almost every commit would be counted as a defect by their metric.

And of course, all of that is data about known defects, and fixed (presumably) defects in the case of commits - how do you compare the number of unknown defects?

There's an underlying assumption [edit: not implying it's yogthos' assumption, of course] that you can simply compare unrelated projects with vastly different policies and processes, and I believe that a lot more manual work is required to get reliable and comparable data.

Post reply on HN