Analysis of longevity of code across many popular projects • Hacker News Board

Re: Analysis of longevity of code across many popular projects

#21

Post by inputcoffee » Tue, Dec 06, 2016, 1:55 PM UTC

I really like this. Suppose we were to accept the suggestion that perhaps linearly decaying code is better built, and more robust, than exponentially decaying code.

Would this give newbies a great new tool to answer their question of which framework or language to learn?

Rails or Django? Django lasts longer. Angular or React? VueJs just a trend?

You could answer all these questions with this kind of analysis.

If someone wants to make a genuine contribution, a blog post contrasting the various decays of Javascript frameworks would be a hit.

Re: Analysis of longevity of code across many popular projects

#22

Post by techiferous » Tue, Dec 06, 2016, 2:00 PM UTC

jsjohnst wrote: post #3

I'd posit that the reason is a nuance of #2, more thought was put into older code on the design of how it should work before making it work. Now we write code so fast that we have to scrap it all and do it again a second time to fix the mistakes of the first time [0]. I'm of the mind that upfront planning would've likely taken less time, but that's simply my opinion and I don't have anything to back it up besides ane…

> The current practice of "move fast and break things" very well could be a better approach.

It's all about context. What's the cost of "breaking things"? Does your not-yet-monetized social network startup go down for a couple of hours? Or does someone die?

Also, I have witnessed first-hand the slowdown in productivity that a "move fast and break things" approach has when what you are breaking is your team's ability to work quickly and confidently with the code base.

Re: Analysis of longevity of code across many popular projects

#23

Post by BlackFly » Tue, Dec 06, 2016, 2:01 PM UTC

cyphar wrote: post #15

It looks like the exponential model isn't a good fit at all -- in all cases it undershoots the decay at the start of the graph and overshoots at the tail end. So while it might "look close" there is some systematic that your model doesn't account for. In particular, I don't agree that all code in a codebase has a constant risk of being replaced -- most projects have different components that are developed at differen…

Agreed; you can reject exponential decay a priori: a code base has some minimal set of functionality that it specifies and there is some minimal set of lines required to provide that functionality. If you believe this, then the "decay" curves, must illustrate an asymptote. The asymptote only gets reduced by breaking backwards compatibility. Such an action would include projects like Angular that go ahead and throw away a ton of core functionality in moving from 1.x to 2.x.

The decay isn't actually decay at all, but represents the complement of lines that define peripheral functionality. Lines defining peripheral functionality typically require modification (refactoring) as additional functionality is added. The asymptote which all the curves illustrate but the fit cannot capture represents the proportion of irreducible core functionality.

Simply adding a constant to the fit might fix all the problems.

Re: Analysis of longevity of code across many popular projects

#24

Post by siscia » Tue, Dec 06, 2016, 2:19 PM UTC

aamederen wrote: post #2

Just Amazing. I wonder if there any research articles discussing the correlation between code-change and other metrics like product quality, change frequency of team members, estimation success, etc.

This is not a research article? What is the difference between this article and one printed in PDF with more complex words and uglier images?

My question sparked for your request of a RESEARCH article and not a "normal" blog post.

I would like to know why you want a "research" article, that I assumed being an academic article, instead of a blog post.

Re: Analysis of longevity of code across many popular projects

#25

Post by spawarotti » Tue, Dec 06, 2016, 2:23 PM UTC

aamederen wrote: post #2

Just Amazing. I wonder if there any research articles discussing the correlation between code-change and other metrics like product quality, change frequency of team members, estimation success, etc.

See the paper "Change bursts as defect predictors" by Nagappan et al. and other papers by the authors.

Re: Analysis of longevity of code across many popular projects

#26

Post by Nomentatus » Tue, Dec 06, 2016, 2:25 PM UTC

BlackFly wrote: post #23

cyphar wrote: post #15

It looks like the exponential model isn't a good fit at all -- in all cases it undershoots the decay at the start of the graph and overshoots at the tail end. So while it might "look close" there is some systematic that your model doesn't account for. In particular, I don't agree that all code in a codebase has a constant risk of being replaced -- most projects have different components that are developed at differen…

Agreed; you can reject exponential decay a priori: a code base has some minimal set of functionality that it specifies and there is some minimal set of lines required to provide that functionality. If you believe this, then the "decay" curves, must illustrate an asymptote. The asymptote only gets reduced by breaking backwards compatibility. Such an action would include projects like Angular that go ahead and throw aw…

This is a close fit to Fechner's Law (not Weber's) relating perceived intensity to stimulus in animal vision. Also, thanks so much, author.

Re: Analysis of longevity of code across many popular projects

#27

Post by spawarotti » Tue, Dec 06, 2016, 2:27 PM UTC

siscia wrote: post #24

aamederen wrote: post #2

Just Amazing. I wonder if there any research articles discussing the correlation between code-change and other metrics like product quality, change frequency of team members, estimation success, etc.

This is not a research article? What is the difference between this article and one printed in PDF with more complex words and uglier images? My question sparked for your request of a RESEARCH article and not a "normal" blog post. I would like to know why you want a "research" article, that I assumed being an academic article, instead of a blog post.

Academic research articles have to follow a quality standard enforced by a peer review process.

Re: Analysis of longevity of code across many popular projects

#28

Post by georgeecollins » Tue, Dec 06, 2016, 2:37 PM UTC

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow.

How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a mass where like a black hole it collapses from some uncomprehensible failure?

There have never been things like this that just grow in complexity forever.

Re: Analysis of longevity of code across many popular projects

#29

Post by siscia » Tue, Dec 06, 2016, 2:47 PM UTC

spawarotti wrote: post #27

siscia wrote: post #24

Earlier quoted context omitted.

This is not a research article? What is the difference between this article and one printed in PDF with more complex words and uglier images? My question sparked for your request of a RESEARCH article and not a "normal" blog post. I would like to know why you want a "research" article, that I assumed being an academic article, instead of a blog post.

Academic research articles have to follow a quality standard enforced by a peer review process.

Sure, but peer review is exactly what we are doing here now.

We are openly discussing the article, we are deciding what we do like and what we don't like about it and we are deciding if the article is worth our attention.

How it is different from an academic peer review process?

Sure the academic peer review processes are blind, but those are a double edge sword.

Re: Analysis of longevity of code across many popular projects

#30

Post by rmchugh » Tue, Dec 06, 2016, 2:50 PM UTC

georgeecollins wrote: post #28

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow. How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a…

I think one of the main reasons that something like Linux keeps on growing is that it needs to support more drivers as they come onto the market. So unless you want to remove support for older hardware, you're more or less going to grow endlessly.