61–70 of 110 posts
The linked blog post [0] and the new security marketing page [1] both have a little more detail on what this actually means. Basically, Semmle offers a static analysis tool that operates on your source code as a graph (from what I understand) and points out bugs and security holes in your code. Github is now offering that for free on repos at all tiers. [0] https://github.blog/2019-09-18-securing-software-together/ […
Earlier quoted context omitted.
Semmle is basically datalog over source code. For what it works for, it works nice. But it is not a pancaea. Security vulnerability finding is almost certainly the wrong target for Semmle - I am unsure why they are trying to push that angle. There are much better stories in things like refactoring and understanding. (I say this having overseen a number of deployments for various reasons, some successful, some not)
You’re wrong on that, security teams at the major tech companies love it, especially for variant analysis. Ask your coworkers at Google! One of which recently left to become Semmle’s Chief Security Officer.
I've also met repeatedly about it with all Google customers over the past few months, Both those currently using it, and those that stopped.
Prior to that, I had met with most that used it but stopped, at the point they started using it. There were several large scale attempts/efforts to use semmle in various ways, by various teams.
I'm trying to be as nice as possible here, since, as I said, I think it is a great tool for a lot of things, and I've been a strong supporter for this kind of technology for those cases (for years, in fact, as i'm sure some folks at Semmle can tell you) so I'd rather not burn it all down, which I expect is what would happen if i did a point by point explanation of everything bad about it.
So I will instead reiterate my claim, and take my downvotes ;)
Earlier quoted context omitted.
Am Microsoft. Mountains were moved to make it scale, but that has been achieved. Semmle can scale with work - it just takes a lot of effort and code.
What had to be done to make it scale?
To simplify a bit you can think of most static analysis algorithms in terms of graph problems where nodes are statements and functions and edges are flow of control and calls. On large codebases the amount of edges, nodes, and calculated data is just too big to keep in memory. The trick is to break the graph intelligently into parts, calculate some sort of summary information for each of them, distributing between cpus or computers, move up to the supegraph of graphs and perform higher level calculations on it.
Earlier quoted context omitted.
It's being run frequently across the entire Windows OS repo. I have heard there is more work to be done to make it scale better, but it can scale.
Am Microsoft. Mountains were moved to make it scale, but that has been achieved. Semmle can scale with work - it just takes a lot of effort and code.
Earlier quoted context omitted.
You’re wrong on that, security teams at the major tech companies love it, especially for variant analysis. Ask your coworkers at Google! One of which recently left to become Semmle’s Chief Security Officer.
I know Fermin, quite well, and I know the state of using it at Google. I technically am the one paying for the contract at this point! I've also met repeatedly about it with all Google customers over the past few months, Both those currently using it, and those that stopped. Prior to that, I had met with most that used it but stopped, at the point they started using it. There were several large scale attempts/efforts…
If you are comfortable sharing more, I’d be curious what you found that it struggles with, without burning it to the ground. It’s possible that different security teams use it in different ways, and it might be more suitable depending on your expectations. It’s also possible that the feedback I heard was during the honeymoon period, and practical issues outweigh the utility once you use it more.
But I’ve seen real 0-days found with it, first hand and second hand, so I’m having trouble reconciling that with your account that it’s not useful for security.
The linked blog post [0] and the new security marketing page [1] both have a little more detail on what this actually means. Basically, Semmle offers a static analysis tool that operates on your source code as a graph (from what I understand) and points out bugs and security holes in your code. Github is now offering that for free on repos at all tiers. [0] https://github.blog/2019-09-18-securing-software-together/ […
Semmle is basically datalog over source code. For what it works for, it works nice. But it is not a pancaea. Security vulnerability finding is almost certainly the wrong target for Semmle - I am unsure why they are trying to push that angle. There are much better stories in things like refactoring and understanding. (I say this having overseen a number of deployments for various reasons, some successful, some not)
Static code analysis tooling is never the end all be all for vulnerability research, but it does let you express vulnerability patterns for implementation type vulnerabilities and find them at a mass scale (that is if your rules/queries are legit).
Earlier quoted context omitted.
I've seen coworkers run semmle queries across the entire Windows OS codebase and find hundreds of issues which were/could result in security vulnerabilities. They've also leveraged it for variant analysis. If I'm not mistaken, the security teams are the largest internal users of Semmle at Microsoft. You're right though, it's not a panacea, and it could probably be great for other uses too.
Any idea what the false positive rate is?
Earlier quoted context omitted.
I know Fermin, quite well, and I know the state of using it at Google. I technically am the one paying for the contract at this point! I've also met repeatedly about it with all Google customers over the past few months, Both those currently using it, and those that stopped. Prior to that, I had met with most that used it but stopped, at the point they started using it. There were several large scale attempts/efforts…
Interesting - I’d like your unfiltered take, but I totally understand your position. I’ve heard very positive feedback from one security person at Google, and I don’t know Fermin, but taking a CSO role there would suggest to me that he believes in it. If you are comfortable sharing more, I’d be curious what you found that it struggles with, without burning it to the ground. It’s possible that different security teams…
Sure, but Fermin was also offered a fairly ridiculous amount of money and a serious promotion :)
I mean, he doesn't not believe in it, of course, but i also think most folks would have taken the role in his situation.
IE it's not the kind of offer that really required a lot of faith
I'll try to write a bit more later after i think about how to frame it :)
Earlier quoted context omitted.
I would love to hear more about this data, as everyone I know who has used it for vulnerability finding has very good things to say. Semmle have also demonstrated its capabilities with some high profile examples.
Who have you talked to about it? Outside Mozilla and Microsoft?