Plotting the source code “TODO” history of the most popular open source projects

Interesting to see that most of them are almost always growing. It would be interesting to compare them to some other metrics, such as TODO/lines_of_code or TODO/num_contributors, to compare the TODO's with the size of the project. I guess that as project gets bigger, it also gets more TODO's

Re: Plotting the source code “TODO” history of the most popular open source projects

#5

Post by anoncow » Mon, May 17, 2021, 7:39 AM UTC

The Golang Todo chart is interesting. It has a sharp peak in April 2018. Linux and Swift seem to be the most in number and uniform in growth.

Re: Plotting the source code “TODO” history of the most popular open source projects

#6

Post by valyagolev » Mon, May 17, 2021, 7:42 AM UTC

thenoblesunfish wrote: post #2

Anyone got a (git-based) one liner to get this info for an arbitrary repo?

here you go

  git rebase -i --exec 'ack TODO | wc -l >> log' HEAD~20

(create a temp branch first!, then substitute your starting revision, and save-quit the editor that'll pop up)

Re: Plotting the source code “TODO” history of the most popular open source projects

#7

Post by unwind » Mon, May 17, 2021, 7:44 AM UTC

thenoblesunfish wrote: post #2

Anyone got a (git-based) one liner to get this info for an arbitrary repo?

As a starting point,

    git log --format=format:"%at %H" | sort -nr

gives a list, with the oldest entry first, of just "hash timestamp" pairs, one per line.

You can then use e.g.

    date -I --date='@1620720025'

to convert the timestamp back to a human-readable date in ISO format, i.e. "2021-05-11".

The next step would be to loop over the list, checkout each revision, grep and wc the TODOs, and collect into date buckets. Anyone? :)

Re: Plotting the source code “TODO” history of the most popular open source projects

#8

Post by Kipters » Mon, May 17, 2021, 7:52 AM UTC

I thought Golang's number of TODOs was much much lower than the others until I looked at the scale

Re: Plotting the source code “TODO” history of the most popular open source projects

#9

Post by pizza234 » Mon, May 17, 2021, 8:15 AM UTC

thenoblesunfish wrote: post #2

Anyone got a (git-based) one liner to get this info for an arbitrary repo?

You've asked for it ;)

This prints the years - you can the group and plot them as you wish (it should be fairly easy, but I wrestled enough with git). It's a non-rigorous script (eg. it assumes nobody's email/name includes an `YYYY-MM-DD`-like string, and that filenames don't include the colon character):

    grep -P '\bTODO\b' -n -R -- * | awk -F: '{ system("git blame "$1" -L "$2","$2) }' | perl -lne 'print /(\d{4})-\d\d-\d\d/'

The working is actually fairly simple:

- grep prints filenames and lines

- awk captures the filename and line, and executes a git blame on it

- perl matches the year and prints it

In the Perl matching expression, month and day are not strictly necessary, but disambiguate potential 4-digit numbers in the email/name.

I'm very underwhelmed by the lack of customization of the `git blame` command - `--porcelain` is also uncustomizable, which makes things even uglier.

Note that `git blame` also mishandles some edges (printing "fatal: file [...] has only 1 line").

Re: Plotting the source code “TODO” history of the most popular open source projects

#10

Post by mekkkkkk » Mon, May 17, 2021, 8:30 AM UTC

The growing number is hardly surprising, yet I don't know exactly what the implications are. There are a lot of different categories of TODOs, ranging from harmless ("it would be nice if this was improved at some point") to critical ("this is a really bad solution that needs to be fixed ASAP"). I wonder if these repos has official definitions of what a TODO entails.