Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding Critical Open Source Projects

opensource.googleblog.com

Post reply on HN
Print view View original

71–80 of 102 posts

Re: Finding Critical Open Source Projects

#71

Post by imglorp »

wyldfire wrote: post #67
megous wrote: post #27

Earlier quoted context omitted.

Which of these match the image in the article, where some random Nebraskan is maintaining some obscure dependency?

I don't think it's explicit in the article, but that's a comic (XKCD). Relevant discussion [1] suggests that there's not a specific project referenced by the comic. When I first read this comic, ntpd [2] [3] came to mind. [1] https://www.explainxkcd.com/wiki/index.php/2347:_Dependency#... [2] https://lwn.net/Articles/701222/ [3] https://lwn.net/Articles/713901/

At least NTP is a standard protocol and there are a bunch of cromulent alternatives to the original Mills ntpd. Chrony and ntpsec are both reasonable. If any one of the implementations went away tomorrow, distros would scramble but they'd have a place to go.

Re: Finding Critical Open Source Projects

#72

Post by rhodozelia »

throwaway342422 wrote: post #51
outsomnia wrote: post #20

Earlier quoted context omitted.

Github's downloads don't tell you when FAANG added your project to their internal project tree, which is then deployed in hundreds of millions of devices.

...not to mention all the critical software running on power plants, industry, aircrafts, ships, cars, satellites, core Internet routing... Did i forget military, e.g. weapons?

I don’t see any open source software in power generation. Vxworks on plcs and then osi pi and some horrible HMI software.

One exception is inductive automation ignition HMI which uses java

Re: Finding Critical Open Source Projects

#73

Post by vlovich123 »

Have you heard of the WWII bullet holes problem that Wald solved? This project seems to fall into the trap that Wald’s colleagues fell into. The more popular a project the more critical it becomes when that’s not actually the case. You want to know where the bullet holes you don’t see are because that’s the thing that’s going to take you out. All the bullet holes you’re seeing and surviving are not great but survivable.

Critical to me means “what’s the thing I have to pay attention to or I’ll suffer consequences”. Having a dependency on a non-popular crate is definitely an increasing score of criticality for that dependency. I may have misread but the fatal error in the metric to me is that popularity of a project increases its criticality when it should decrease. A popular project with lots of contributors means it has lots of stake holders already and a way to successfully manage that. What you want is the small independent projects that popular projects depend on. In other words, what’s the smallest, hardest to notice malicious change I can make in the supply chain of software development to enact the most disruptive change?

Re: Finding Critical Open Source Projects

#74

Post by rhencke »

abharya wrote: post #61
throwaway342422 wrote: post #50

Earlier quoted context omitted.

The methodology is pretty silly. It rewards activity and popularity. A lot of critical infrastructure software is not active and not often spoken about! Case in point: The https://www.cip-project.org/faq project highlight the needs of very-long-term support for OS components that run on critical infrastructure such as power stations. The https://www.cip-project.org/faq project is based on Debian. Very little of it is…

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

I would like to see a measure of criticality that takes the following into account:

* Critical projects may have very little activity/maintenance. For example, Bash 4.0 to Bash 5.0 was only 123 commits over 8 years. But, Bash is a absolutely a critical project (ask any org about how much work they had to do when affected by https://en.wikipedia.org/wiki/Shellshock_(software_bug) ).

* A measure of criticality should understand _as many of the various forms of dependence on software_ that may occur that it can. Dependencies can take many forms, such as:

a package manager resolving a dependency

a user purchasing a mobile phone with software pre-installed

a user visiting a website (react/jquery/etc)

etc.

* Criticality should understand if, how, and when dependencies are updated. For example, fixing a bug in Chrome and distributing that fix to 80% of users in 1 week is feasible. Fixing a bug in Bash and distributing that fix to 80% of users in 1 week is not so feasible.

Re: Finding Critical Open Source Projects

#75

Post by rhencke »

vlovich123 wrote: post #73

Have you heard of the WWII bullet holes problem that Wald solved? This project seems to fall into the trap that Wald’s colleagues fell into. The more popular a project the more critical it becomes when that’s not actually the case. You want to know where the bullet holes you don’t see are because that’s the thing that’s going to take you out. All the bullet holes you’re seeing and surviving are not great but survivab…

Your last sentence nails it:

> What’s the smallest, hardest to notice malicious change I can make in the supply chain of software development to enact the most disruptive change?

Re: Finding Critical Open Source Projects

#76

Post by abharya »

vlovich123 wrote: post #73

Have you heard of the WWII bullet holes problem that Wald solved? This project seems to fall into the trap that Wald’s colleagues fell into. The more popular a project the more critical it becomes when that’s not actually the case. You want to know where the bullet holes you don’t see are because that’s the thing that’s going to take you out. All the bullet holes you’re seeing and surviving are not great but survivab…

This is still an early project to start somewhere.

We are working on this problem as well. Trying to identify dependency trees and which smaller projects are the most widely used (indirect deps) and will impact the most on the critical projects in the list.

Re: Finding Critical Open Source Projects

#78

Post by justinclift »

jchw wrote: post #46
justinclift wrote: post #43

Earlier quoted context omitted.

Definitely needs more work. The main Go SQLite3 for accessing SQLite databases is in the top "C" list. But SQLite itself doesn't seem to be included in any of them. o_O

That might be because sqlite isn’t on Github.

Yeah, that could be the case. That being said, lots of projects aren't on GitHub, and SQLite does have a mirror there which is kept up to date:

https://github.com/sqlite/sqlite/

Re: Finding Critical Open Source Projects

#79

Post by justinclift »

abharya wrote: post #58
arp242 wrote: post #47

Earlier quoted context omitted.

That's probably because SQLite doesn't use git, and this tool seems to require git. Actually, in its current state it seems to require GitHub: https://github.com/ossf/criticality_score/blob/main/critical... This probably omits some other projects as well which don't use git or GitHub.

Yes correct. Right now, we are query-ing projects hosted on Github, but will be expanding to our source control system in the near future.

No worries. Just to point out though, SQLite does have an official mirror repo on GitHub:

https://github.com/sqlite/sqlite/

Re: Finding Critical Open Source Projects

#80

Post by vlovich123 »

abharya wrote: post #76
vlovich123 wrote: post #73

Have you heard of the WWII bullet holes problem that Wald solved? This project seems to fall into the trap that Wald’s colleagues fell into. The more popular a project the more critical it becomes when that’s not actually the case. You want to know where the bullet holes you don’t see are because that’s the thing that’s going to take you out. All the bullet holes you’re seeing and surviving are not great but survivab…

This is still an early project to start somewhere. We are working on this problem as well. Trying to identify dependency trees and which smaller projects are the most widely used (indirect deps) and will impact the most on the critical projects in the list.

Don’t get me wrong. I appreciate this work and it’s super important if you get it right. I’m worried the first step is drastically in the wrong philosophical direction as far as the metric itself goes.

If you’re just proving out the tech then do that but the current publishing of the most “critical” projects is laughably wrong (as I’m sure you’ve heard from the comments on the page). The project isn’t a dead end but any attempt to use the data right now to drive decisions is a lost cause. Maybe you have some secret sauce that makes it work better for internal Google projects but if that’s the case that methodology should be published as well so that people can get an understanding of how Google uses this more effectively.

I’m just contrasting this to a paper Google published recently about how to measure uptime more effectively in cloud. Every step of that paper was eminently approachable and didn’t have any obvious philosophical flaws that stood out to me (we can argue about them but any potential arguments were called out).

Again, I applaud the attempt. This is a super important problem and trying to tackle it is laudable. Getting the right metrics is crucial and this is being released far too early with a presentation that makes it seem far more complete than it is.

Post reply on HN