Live data from Hacker News

Analysis of longevity of code across many popular projects

erikbern.com

31–40 of 52 posts

Re: Analysis of longevity of code across many popular projects

#31
post #15

It looks like the exponential model isn't a good fit at all -- in all cases it undershoots the decay at the start of the graph and overshoots at the tail end. So while it might "look close" there is some systematic that your model doesn't account for. In particular, I don't agree that all code in a codebase has a constant risk of being replaced -- most projects have different components that are developed at differen…

Agreed... looking at the graphs, the fit screams "non-exponential" to me.

Re: Analysis of longevity of code across many popular projects

#32

I really like this. Suppose we were to accept the suggestion that perhaps linearly decaying code is better built, and more robust, than exponentially decaying code. Would this give newbies a great new tool to answer their question of which framework or language to learn? Rails or Django? Django lasts longer. Angular or React? VueJs just a trend? You could answer all these questions with this kind of analysis. If some…

Ditto. That would be useful to know as newbie, I'm hoping someone can give an answer or atleast point into the right direction.

Re: Analysis of longevity of code across many popular projects

#33
post #15

It looks like the exponential model isn't a good fit at all -- in all cases it undershoots the decay at the start of the graph and overshoots at the tail end. So while it might "look close" there is some systematic that your model doesn't account for. In particular, I don't agree that all code in a codebase has a constant risk of being replaced -- most projects have different components that are developed at differen…

It might be useful to fit an exponential for each years code. The oldest code seems to decay to some constant and then linger forever at some LOC - it does not decay to zero LOC. It may be that new code is inherently inefficient and over time is gets cleaned up. Also some functionality may be abstracted and built into new code and removed from the old. Over time you end up with a chunk of code doing its one thing and doing it well. This is all speculation of course. I hope this gets looked at a lot more.

Re: Analysis of longevity of code across many popular projects

#35
post #30

Look at how consistently the lines of code grow for these projects. I doubt that is surprising but think about the implications. Linux is a pretty old open source project and still on balance the lines of code just grow. How many lines of code will it be in fifty years? Will we have to come up with new systems to manage the fact that individuals only really understand smaller and smaller pieces of it? Will it reach a…

I think one of the main reasons that something like Linux keeps on growing is that it needs to support more drivers as they come onto the market. So unless you want to remove support for older hardware, you're more or less going to grow endlessly.

Right, but nothing can grow endlessly. Yet these projects do grow relentlessly, almost never getting any smaller, for as long as we have kept track. It can't go on forever, so how can it end?

Re: Analysis of longevity of code across many popular projects

#36
post #30

Earlier quoted context omitted.

I think one of the main reasons that something like Linux keeps on growing is that it needs to support more drivers as they come onto the market. So unless you want to remove support for older hardware, you're more or less going to grow endlessly.

Right, but nothing can grow endlessly. Yet these projects do grow relentlessly, almost never getting any smaller, for as long as we have kept track. It can't go on forever, so how can it end?

The industry could converge on standards which allow certain classes of "repeated but different" code (drivers?) to collapse. By collapse, I mean they could share a generic version of code, thus reducing the overall line count.

Or a new implementation (or OS) could come along, with compelling advances and no baggage. The old code doesn't so much collapse as become obsolete.

Re: Analysis of longevity of code across many popular projects

#37

This is awesome. I ran it on pyramid for fun. http://imgur.com/a/KZ9KR

A catastrophe based model seems more appropriate. You see big refactoring taking place. Only in a big number sum those Poisson-like events turn into exponentials.

2010 was a major transition for pyramid as it was re-branded from repoze.bfg and merged with the pylons project.

Re: Analysis of longevity of code across many popular projects

#38
post #30

Earlier quoted context omitted.

I think one of the main reasons that something like Linux keeps on growing is that it needs to support more drivers as they come onto the market. So unless you want to remove support for older hardware, you're more or less going to grow endlessly.

Right, but nothing can grow endlessly. Yet these projects do grow relentlessly, almost never getting any smaller, for as long as we have kept track. It can't go on forever, so how can it end?

Why can't it go on forever? ("Forever" meaning as long as computers and humans exist.) I don't see any particular reason why.

Re: Analysis of longevity of code across many popular projects

#39
A lot of projects I've worked on have utility libraries consisting of mostly stateless, pure functions -- I have a theory that these constitute some of the longest lived code. That, and database models, which tend to be easier to expand than to contract. I'd be curious to see some analysis along these lines.
Post reply on HN