Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding Critical Open Source Projects

opensource.googleblog.com

Post reply on HN
Print view View original

91–100 of 102 posts

Re: Finding Critical Open Source Projects

#91

Post by orf »

abharya wrote: post #15
eh78ssxv2f wrote: post #11

I don't understand how gnucash is more critical than Arduino, wine or nginx. Is gnucash very widely used?

It does seem to be popular w.r.t user downloads and other github metrics. E.g. https://sourceforge.net/projects/gnucash/files/stats/timelin... Wikipedia - "As of July 2018, SourceForge shows a count of over 6.3 million downloads of the stable releases starting from November 1999[24] Also, Sourceforge shows that current downloads are running at ~7,000 per week.[25] This does not include other software download sites a…

Man... that’s 700 downloads a day. 29 downloads an hour. That’s by no means “popular”. At least not in the scale you should be looking at.

Re: Finding Critical Open Source Projects

#92

Post by abharya »

rwmj wrote: post #25

It's nice that they're automating this even though the results seem currently a bit random. Fedora has a concept of Critical Path packages ( https://fedoraproject.org/wiki/Critical_path_package ). The concept is related to various activities, eg. a package is critical if is is needed when installing a graphical desktop. But as far as I know packages are picked manually and therefore we probably miss packages or over/…

Thanks for feedback, adding it for analysis in https://github.com/ossf/criticality_score/issues/23

Re: Finding Critical Open Source Projects

#93

Post by abharya »

jcelerier wrote: post #53
abharya wrote: post #18

Earlier quoted context omitted.

World is not perfect. In case of gcc, it is only a mirror on github [ https://github.com/gcc-mirror/gcc ], so we dont get all the metrics, we do plan to improve this part [but this will be slow to clone repo, know their custom issue trackers, etc]. Gnucash is still a popular project, has like ~7k downloads a week, see wikipedia page.

> Gnucash is still a popular project, has like ~7k downloads a week, see wikipedia page. omg they're actually doubling down on this. Libreoffice has something like 50k downloads per day and isn't listed, while among "critical projects" there is... minetest ? tesseract ? battle for wesnoth ? dolphin ? citra ? There's qt creator (and a ton of Qt-using projects) but not Qt ?

Thanks for your feedback, we do want to fix this.

And we do agree that there are several critical packages missing that don't use all metrics on github, plus non-github projects. Capturing this in https://github.com/ossf/criticality_score/issues/21

Re: Finding Critical Open Source Projects

#94

Post by lkcl »

abharya wrote: post #61
throwaway342422 wrote: post #50

Earlier quoted context omitted.

The methodology is pretty silly. It rewards activity and popularity. A lot of critical infrastructure software is not active and not often spoken about! Case in point: The https://www.cip-project.org/faq project highlight the needs of very-long-term support for OS components that run on critical infrastructure such as power stations. The https://www.cip-project.org/faq project is based on Debian. Very little of it is…

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

one of the things that's important is not to include google's lawyers bias against the GPL license. i always wondered where the bias against the GPL came from, within google, and learned of the existence of the in-house legal team. it turns out that they have been advising google employees for some considerable time, "avoid the GPL, avoid the GPL".

unfortunately, as legal advice, those google employees (right the way to management) do not have the backbone to say, "err no actually, GPL-licensed code is the critically strategically important leveller that forces aberrant companies to collaborate rather than sponge off of underfunded projects".

Re: Finding Critical Open Source Projects

#95

Post by lkcl »

rhencke wrote: post #64
abharya wrote: post #61

Earlier quoted context omitted.

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

I strongly recommend you use the packages maintained in Linux distributions as a means for discovery. They're well-organized and maintained and easily accessible programmatically - you can even parse the package dependencies programmatically, as well as have full access to the original source code.

good advice. "apt-get install apt-rdepends" and it becomes possible to work out the reverse-dependencies of packages.

by counting the numbers it becomes pretty blindingly obvious what the critical dependencies are. as mentioned in another post above, bash and glibc6 are blindingly-obviously high on the list... yet the GNU Project receives an unbelievably low amount of funding despite their critical importance.

likewise, this particular bug in binutils ld, which centres around the incredibly short-sighted "4GB should be enough for anyone" removal of Dr Stallman's memory-resident algorithms in the late 90s, is having some very serious consequences:

https://sourceware.org/bugzilla/show_bug.cgi?id=22831

yet because there's no money not even from redhat nobody's looking at it.

likewise: PAM no longer has a proper maintainer, and hasn't had for... a decade?

these are projects that people are relying on yet completely forgetting they're a critical part of the infrastructure!

why? because, just as rhencke said above: they're not on github, they've not got "unnecessary changes" which are counted as "activity to be glorified and worshipped".

abharya: i heard on slashdot the intent to start from github, to exclusively focus on github. this will turn out to be a serious mistake.

Re: Finding Critical Open Source Projects

#96

Post by lkcl »

abharya wrote: post #61
throwaway342422 wrote: post #50

Earlier quoted context omitted.

The methodology is pretty silly. It rewards activity and popularity. A lot of critical infrastructure software is not active and not often spoken about! Case in point: The https://www.cip-project.org/faq project highlight the needs of very-long-term support for OS components that run on critical infrastructure such as power stations. The https://www.cip-project.org/faq project is based on Debian. Very little of it is…

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

people. as a computer scientist you're probably thinking, "this can be solved by analysing a source code forge" or, "this can be solved by running an algorithm". it can't (or, more to the point: it can tell you quantities, but not quality or value).

i mentioned in another post: github "glorifies" the person and the changes that they make. "look at mee! look at mee! i'm making a commit! i'm wiping my backside now! aren't i great!" which gets you precisely zip in terms of actual strategic value.

changes measure change.

people will tell you - if you let them - by providing you with the information needed to make a qualitative assessment.

so.

provide a type of wiki/website that allows qualitative assessments to be made, on a per-library / per-project basis. then put the metrics (the "criticality value") onto that.

pre-seed that wiki/website with stuff from github if you feel so inclined but DO NOT limit the wiki/website to exclusively github. i repeat again: doing so would be a disastrous mistake.

Re: Finding Critical Open Source Projects

#97

Post by lkcl »

mbakke wrote: post #56
hn_throwaway_99 wrote: post #10

As others have mentioned, while this may seem like a good idea, the results are often bizarre, and it's not hard to see why - the metrics and algorithm are here: https://github.com/ossf/criticality_score#criticality-score . That algorithm seems unnecessarily complicated and includes somewhat dubious metrics when, in my mind, the only thing that really "counts" when it comes to "criticality" are "how many other things…

One cool feature of functional package managers such as Nix and Guix is that the dependency graphs are entirely transparent and can be inspected programmatically. I wrote a script that lists the number of dependents for each package in Guix by traversing the package graphs: https://gist.github.com/mbakke/f354272666fbef09c5229f7b85377... Running it takes about 16 seconds on my laptop, and piping to 'grep -v bootstrap…

analysing a source-only-based distro (one where the compiler is required as a dependency) should make the importance of the compiler(s) clearer.

source-only distros include macports, gentoo, and so on.

Re: Finding Critical Open Source Projects

#98

Post by lkcl »

btw, just one thing, having seen this slashdot comment: https://news.slashdot.org/comments.pl?sid=17816088&cid=60823...

what i strongly recommend that google do - instead of doing this work which is, as you've probably noticed from the comments, well-meaning but highly likely to be biased - is:

    *make a large donation to NLnet*
don't tell them what to do with the money: leave that up to them. NLnet is extremely good at ensuring that money is used effectively, by requiring that they come up with a project plan involving milestones. they do not just "dump money at the developer", which has a known high historically-backed probability of causing more harm than good: they require the milestones to be completed, 100%, before the money is paid.

note, here: they do not use algorithms to assess a project: they use people. they also assess the proposal against the usefulness of achieving key objectives.

Re: Finding Critical Open Source Projects

#100

Post by abharya »

lkcl wrote: post #95
rhencke wrote: post #64

Earlier quoted context omitted.

I strongly recommend you use the packages maintained in Linux distributions as a means for discovery. They're well-organized and maintained and easily accessible programmatically - you can even parse the package dependencies programmatically, as well as have full access to the original source code.

good advice. "apt-get install apt-rdepends" and it becomes possible to work out the reverse-dependencies of packages. by counting the numbers it becomes pretty blindingly obvious what the critical dependencies are. as mentioned in another post above, bash and glibc6 are blindingly-obviously high on the list... yet the GNU Project receives an unbelievably low amount of funding despite their critical importance. likewi…

There is no exclusive focus, we are just starting somewhere where we can see the various metrics. Plan is to expand to non-github projects and other places (like custom issue trackers), but this is not straightforward as it sounds. Ideas welcome!. https://github.com/ossf/criticality_score/issues/29
Post reply on HN