Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analysis of longevity of code across many popular projects

erikbern.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Analysis of longevity of code across many popular projects

#41

Post by JoachimSchipper »

georgeecollins wrote: post #28

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow. How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a…

Linux already relies on specialized tools - like Cocinelle=spatch - to migrate very large codebases at once. Linux has also grown from a hobbyist project into a fairly complex (informal!) organization of highly-skilled full-time developers, just to keep development manageable.

Re: Analysis of longevity of code across many popular projects

#42

Post by jtigger »

I wish we could capture the "inventiveness" of a particular project — how well the problem was understood when the project initiated.

There had been _many_ *nix'es by 2006, so the territory had significant prior art and with it collective deep understanding of the problems being solved. Angular sprouted alongside a number of other SPA frameworks in an ecosystem that was experiencing a "growth-spurt" (using that term loosely) — lots of variables.

Re: Analysis of longevity of code across many popular projects

#43

Post by jtigger »

Another feature of the model could be stability of product vision. Is there a correlation between the half-life of committer membership and that of the code? How has the problem space of the product changed over time?

Perhaps we could talk about "intrinsic churn" vs. "accidental churn". The former results from the codebase keeping up with the "drift" in the problem space; the latter comes from having to learn.

Re: Analysis of longevity of code across many popular projects

#44

Post by jtigger »

msluyter wrote: post #39

A lot of projects I've worked on have utility libraries consisting of mostly stateless, pure functions -- I have a theory that these constitute some of the longest lived code. That, and database models, which tend to be easier to expand than to contract. I'd be curious to see some analysis along these lines.

Nice hypothesis! Reminds me of the shearing layers from Foote & Yoder's "Big Ball of Mud" — http://www.laputan.org/mud/mud.html#ShearingLayers

Re: Analysis of longevity of code across many popular projects

#45

Post by jtigger »

georgeecollins wrote: post #28

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow. How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a…

> There have never been things like this that just grow in complexity forever.

The evolution of the species? Homo sapiens seem rather complex.

Perhaps what we'll see is that over time, large portions of what have been long list of pedantic instructions become decision models that get tweaked over time. Imagine when referring to the size of a software system using two numbers: MLOCs and MDPs (Matured Decision Points). Not unlike industrial revolution assembly lines compared to the automated factories of today — modern factory workers are some degrees removed from the step-by-step movements.

Re: Analysis of longevity of code across many popular projects

#46

Post by skeltoac »

Code from year one may still be the same code but when it gets moved or reformatted its cohort is updated. If a bad change is reverted, will the cohort for those lines of code also be reverted? The effect of understated longevity is not so obvious when it is gradual and organic. Sometimes an event in a project's history makes the effect very obvious.

https://blog.yoavfarhi.com/2016/12/06/half-life-wordpress-co...

Re: Analysis of longevity of code across many popular projects

#48

Post by aamederen »

siscia wrote: post #24
aamederen wrote: post #2

Just Amazing. I wonder if there any research articles discussing the correlation between code-change and other metrics like product quality, change frequency of team members, estimation success, etc.

This is not a research article? What is the difference between this article and one printed in PDF with more complex words and uglier images? My question sparked for your request of a RESEARCH article and not a "normal" blog post. I would like to know why you want a "research" article, that I assumed being an academic article, instead of a blog post.

What I interested in was not the "research article" part but the comparison case.

This article comes up with a tool measuring the half-life of the code and demonstrates it on some project. What I was requested as an addition is the discussion of correlation of this metric with other metrics.

Being said that, a "paper" makes a difference compared to a "blog post" at some cases. Sometimes, in order to convince your directors and project managers about your change proposals to the programming processes, you need to support your idea with more serious work.

For example, in my previous company, I could use such an academic research in order to demand more time budget for "code-cleanup" periods where the team focuses just to re-writing the parts of legacy code, instead of bugfixes and new features.

I am surprised that this small request offended someone.

Re: Analysis of longevity of code across many popular projects

#49

Post by buzzybee »

georgeecollins wrote: post #28

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow. How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a…

There's a pretty definite threshold, perhaps somewhere in the early 10,000's of lines(depending on the language and how densely you wrote it), where the system becomes bigger than your headspace, and at that point, I think, the real challenge for code reduction begins. And there are some very big wins possible by consistently reducing the problem into "language for solving the problem plus trivial code in that language" - VPRI's STEPS articles [0] demonstrate as much.

But it's perceptually less work to accrete a few hundred new lines to the system than to engineer a 10x reduction that introduces an entirely new architecture layer, so nobody does, or only a few research-driven projects. Programmers are, after all, lazy and deadline-driven. And while proprietary software tends to accrete due to corporate dynamics creating an environment of continuous growth, open-source projects are often built on accretion as an assumed benefit.

My own software goal, having viewed this landscape, is to be stubborn enough for long enough to score some of the 10x wins and share those so that "more with less" becomes a little more viable. I do think that all codebases go obsolete eventually, but the overall ecosystem doesn't have to depend on them so long there's data and protocol compatibility.

[0] http://vpri.org/html/writings.php

Post reply on HN