Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding Critical Open Source Projects

opensource.googleblog.com

Post reply on HN
Print view View original

81–90 of 102 posts

Re: Finding Critical Open Source Projects

#81

Post by arp242 »

abharya wrote: post #58
arp242 wrote: post #47

Earlier quoted context omitted.

That's probably because SQLite doesn't use git, and this tool seems to require git. Actually, in its current state it seems to require GitHub: https://github.com/ossf/criticality_score/blob/main/critical... This probably omits some other projects as well which don't use git or GitHub.

Yes correct. Right now, we are query-ing projects hosted on Github, but will be expanding to our source control system in the near future.

It would be nice if this could be mentioned a bit clearer in the blog post and/or README; it's not really that obvious at all and I had to go to the source to check, and loads of people here seem confused about it since it more or less implies "we looked at all open source projects".

Re: Finding Critical Open Source Projects

#82

Post by abharya »

justinclift wrote: post #78
jchw wrote: post #46

Earlier quoted context omitted.

That might be because sqlite isn’t on Github.

Yeah, that could be the case. That being said, lots of projects aren't on GitHub, and SQLite does have a mirror there which is kept up to date: https://github.com/sqlite/sqlite/

The issue with the mirror is we don't get the important stats to make decisions. E.g. number of contributors, issue changes due to custom issue tracker. We are still thinking on how to add information from such cases in automated fashion, ideas welcome!

Re: Finding Critical Open Source Projects

#83

Post by abharya »

TAForObvReasons wrote: post #37
Buetol wrote: post #9

Top 10: - Python: salt, core ( https://github.com/home-assistant/core ), pandas, scikit-learn, numpy, airflow, erpnext, matplotlib, pytest & pip - Rust: servo, cargo, rust-clippy, tokio, rust-analyzer, tock, tikv, alacritty, libc & substrate - JS: node, react-native, react, gatsby, three.js, bootstrap, material-ui, odoo, next.js & Rocket.Chat - Java: elasticsearch, flink, spring-boot, hadoop, netty, jenkins, beam, ba…

Some de-duplication is needed. 12, 13, and 14 for JS are amphtml

That is a bug, will be fixed soon.

Re: Finding Critical Open Source Projects

#84

Post by abharya »

stonogo wrote: post #30
Buetol wrote: post #9

Top 10: - Python: salt, core ( https://github.com/home-assistant/core ), pandas, scikit-learn, numpy, airflow, erpnext, matplotlib, pytest & pip - Rust: servo, cargo, rust-clippy, tokio, rust-analyzer, tock, tikv, alacritty, libc & substrate - JS: node, react-native, react, gatsby, three.js, bootstrap, material-ui, odoo, next.js & Rocket.Chat - Java: elasticsearch, flink, spring-boot, hadoop, netty, jenkins, beam, ba…

I'm interested in the gulf between low-level systems code and effectively end-user code here. Things like GLib, Cairo, Harbuzz, etc -- none of the end-user tools work without them. It really demonstrates just how difficult this evaluation can be.

It is definitely difficult especially with critical dependencies. We are looking for any criteria to identify these in automated fashion. Parallely, for ones we know about those are we are trying to run our automated tools on. E.g. glib, cairo, harfbuzz are all continuously fuzzed as part of OSS-Fuzz - https://github.com/google/oss-fuzz/tree/master/projects

Re: Finding Critical Open Source Projects

#85

Post by abharya »

schoen wrote: post #24
abharya wrote: post #15

Earlier quoted context omitted.

It does seem to be popular w.r.t user downloads and other github metrics. E.g. https://sourceforge.net/projects/gnucash/files/stats/timelin... Wikipedia - "As of July 2018, SourceForge shows a count of over 6.3 million downloads of the stable releases starting from November 1999[24] Also, Sourceforge shows that current downloads are running at ~7,000 per week.[25] This does not include other software download sites a…

> This does not include other software download sites as well as Linux distributions that provide download from their own repositories. Isn't that a pretty significant difficulty for the interpretation of this number, though? Couldn't there be some package that is installed by default in, say, Ubuntu (and that Ubuntu can't boot without), so therefore millions of users are using it -- but perhaps the "downloads" seen…

coreutils is in the top 200 list, c_top_200.csv:coreutils,https://github.com/coreutils/coreutils,C,90,0,240,5,2.2,50,1...

There will always be edge cases and scenarios we are not taking into account, please provide feedback on issue tracker and provide any suggestions so we can account these.

Re: Finding Critical Open Source Projects

#86

Post by abharya »

rhencke wrote: post #74
abharya wrote: post #61

Earlier quoted context omitted.

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

I would like to see a measure of criticality that takes the following into account: * Critical projects may have very little activity/maintenance. For example, Bash 4.0 to Bash 5.0 was only 123 commits over 8 years. But, Bash is a absolutely a critical project (ask any org about how much work they had to do when affected by https://en.wikipedia.org/wiki/Shellshock_(software_bug) ). * A measure of criticality should u…

filed https://github.com/ossf/criticality_score/issues/23

Re: Finding Critical Open Source Projects

#87

Post by abharya »

detaro wrote: post #31
abharya wrote: post #12

Earlier quoted context omitted.

- Other metrics such as how many contributors and organizations are involved, how many user feature requests and bugs getting reported, those are all important project importance and not just "dependency count". some projects can be standalone, so as per your algo, those should be very low. - Downloads data is not available for most repos, please find a reliable metric to use. - Package repo dependencies works, but i…

So basically the algorithm is designed to not find projects that are critical because they are deep in the foundations, depended on by nearly everyone but only worked on by a few people? (which is what "critical" would suggest at least to me) This seems to be mostly a "github marketing index"... I seriously hope no actual decisions about resource allocation etc are made based on this.

We are working on this problem, it is not simple. Identifying dependency trees reliably across languages is not straightforward [only nice for package manager ones]. Follow https://github.com/ossf/criticality_score/issues/8

Re: Finding Critical Open Source Projects

#88

Post by justinclift »

abharya wrote: post #82
justinclift wrote: post #78

Earlier quoted context omitted.

Yeah, that could be the case. That being said, lots of projects aren't on GitHub, and SQLite does have a mirror there which is kept up to date: https://github.com/sqlite/sqlite/

The issue with the mirror is we don't get the important stats to make decisions. E.g. number of contributors, issue changes due to custom issue tracker. We are still thinking on how to add information from such cases in automated fashion, ideas welcome!

Would it be feasible to add support for Fossil, so it's not just a git-only tool?

Re: Finding Critical Open Source Projects

#89

Post by gfxgirl »

commandlinefan wrote: post #69
megous wrote: post #27

Earlier quoted context omitted.

Which of these match the image in the article, where some random Nebraskan is maintaining some obscure dependency?

That would be "left-pad": https://qz.com/646467/how-one-programmer-broke-the-internet-...

Yes, because the maintenance of such a library is astronomical!

Re: Finding Critical Open Source Projects

#90

Post by gfxgirl »

hn_throwaway_99 wrote: post #10

As others have mentioned, while this may seem like a good idea, the results are often bizarre, and it's not hard to see why - the metrics and algorithm are here: https://github.com/ossf/criticality_score#criticality-score . That algorithm seems unnecessarily complicated and includes somewhat dubious metrics when, in my mind, the only thing that really "counts" when it comes to "criticality" are "how many other things…

Maybe I mis-understood what's supposed to be being checked here. I thought the point was surfacing critical projects that are under supported.

So a highly dependent project that has plenty of support for its needs should not need to be brought to anyone's attention. I'm guessing python, llvm, clang, chromium, webkit, C#, visual studio code, off the top of my head are all well funded and supported open source projects.

I don't know what good examples of under supported but critical open source projects are. I guess I've read that OpenSSL was massively under supported but apparently the solution chosen was to throw it under the bus and promote libssl or boringssl or something like that (completely out of my expertise)

Post reply on HN