Well, this is a real circus.
I have tested and deployed several scanning tool (and have contributed to nessus some years ago - their scanning capacities were really mediocre compared to the precision of the results. In out 100+k IPs on 10.x networks the time to scan was horrendous. But once an IP was scanned, the results were fine). A small disclaimer: I use several solutions but have no stakes in any of them (neither company nor personal). I have build relationships with them that helps me to provide substantiated feedback that sometimes event get taken into account :)
There are several issues, none of them has an automated way of fixing.
First there is the criticality of the finding. We found that often (especially on Linux) you will get a "raw" critical issue on a component that was not patched; And why it was not patched? Because the vendor set a low level.
It is then that the discussion between the security team and the OS one starts - I have experimentally found that the vendor rating is usually better because they take into account the actual context of the deployment, especially for libraries.
OTOH crowdstrike vulnerability rating (I forgot the name of the component, it is on top of the EDR) is actually two ratings: the CVSS-reated one, and their own where they look at actual exploits and deployment. I have decided that critical+critical = actually critical that must be patched no matter what. This is the only way I found to quantitatively auto-decide on important patches. The goal is of course to patch everything but the reality is that there are so many loose ends in products that fighting for anything below the top two levels does not make sense.
The very sad thing here is that this is perceived as an issue of the OS/middleware and not the shitty products vendors deploy. They use some undocumented or weak solutions and then cry when everything breaks down. I develop myself (amateur developer) and have never had any issues for 30 years when upgrading the OS because I actually care about following the docs.
The second problem is that today products have nested dependencies. For our own products we do an awfully time taking and precise review of the actual impact on the product but this is because people actually care (this is not my team so I ma not taking any credit - I just like their approach). When a customer comes with a question we have a comprehensive response. This works or not, some people when they see "vulnerability" they stop at that and want that word to disappear.
When we are on the receiving side of such products (and log4j is a perfect example), we ask the vendor. If this is open source the answer is usually very good, there are some github/gitlab issues that address the point. Commercial vendors are much worse because they either say "no worries" or take weeks to respond. When a "no worry" comment comes from a company that cannot explain why they limit their passwords to 27 characters, it is worrisome - a bit like when my children say "no worries" (this is the moment when I start to worry).
Scanners are quite slow to respond to non obvious vulnerabilities (the ones you cannot easily tag though OVAL). For log4j for instance I coded a scanner/reflector on the night of Thursday (the information popped up about noon on Thursday western europe time). The scanners followed up after a week, but one could fine good ones open source quickly as well. So my advice would be to closely monitor GitHub for such tools when there is pressure (we switched to another tool for log4j when my team and the OS ones were successively finding better ones). This also led us to investigate a fantastic tool/framework to write scanners, which name escapes me right now but I can comment here when I am back from vacation and get hold of the gal who is looking at that.
The TL;DR version is that
- a few tools provide a way to quantify the risk (that's a huge slit but the best I can do) - at least there is a reasonable chance that you will not miss the worst ones
- GitHub activity when shit hits the fan is extraordinarily useful - the best tools emerge there, before the vendors catch up
- vendors are medium, but is is not as bad as you read
- your CISO should make the effort to communicate that there will be availability issues when the IT/security team decides to switch off a key component (say - email) if this presents a risk for the company. I have the chance to have a CEO who not only understands this but actually challenged me once to why I did not cut earlier. Having that arranged upfront talks a huge stress off the security team.
- and finally the sad reality is that the vast, vast majority of issues we get are from users who click on stuff
I am sure I forgot a lot - let me know if I can help with some details.