71–80 of 110 posts
Earlier quoted context omitted.
Any idea what the false positive rate is?
If you use the out of the box rules for any of these tools the false positive rate will usually be pretty high. The trick is to write custom rules that are more tailored to your code.
Earlier quoted context omitted.
You’re wrong on that, security teams at the major tech companies love it, especially for variant analysis. Ask your coworkers at Google! One of which recently left to become Semmle’s Chief Security Officer.
First of all, Daniel Berlin is pretty senior at a reasonably large tech company a lot of us here have heard of. Secondly, I know Microsoft loves it, which is presumably where your telemetry comes from, and I know a lot of security people on Twitter are fans of the technology, but I've been asking around and "love it" is not the signal I'm getting from software security blue team people. "I installed it, I guess it do…
I mean think about it, if you wanted to write a query against the AST of a target, would you find that useful? Or in a given codebase, if you find one bug, would you like the ability to capture that in a query that can tell you if a similar mistake was made elsewhere?
Out of the box, it isn’t going to give you much value. It’s the power of the query language, if it’s your job to do that, where you’ll see the benefits.
But don’t take my word for it, just try it out.
Their licensing model may be problematic for your use case though. I only vaguely understand what you do, but last I asked them about it, it’s not possible to get a personal license that a security person can use for multiple projects, and my read was that they had no interest in selling to individuals anytime soon.
Earlier quoted context omitted.
Interesting - I’d like your unfiltered take, but I totally understand your position. I’ve heard very positive feedback from one security person at Google, and I don’t know Fermin, but taking a CSO role there would suggest to me that he believes in it. If you are comfortable sharing more, I’d be curious what you found that it struggles with, without burning it to the ground. It’s possible that different security teams…
"and I don’t know Fermin, but taking a CSO role there would suggest to me that he believes in it." Sure, but Fermin was also offered a fairly ridiculous amount of money and a serious promotion :) I mean, he doesn't not believe in it, of course, but i also think most folks would have taken the role in his situation. IE it's not the kind of offer that really required a lot of faith I'll try to write a bit more later af…
If your data is closer to a model where security bug hunters whose sole job is to find vulnerabilities and audit code, and it was deemed not useful in that scenario, then yes, I am at odds with your claim. Admittedly, that’s a pretty niche set of customers. If you don’t learn Semmle QL, and you aren’t writing queries, it’s probably not for you.
Earlier quoted context omitted.
If you use the out of the box rules for any of these tools the false positive rate will usually be pretty high. The trick is to write custom rules that are more tailored to your code.
So how much is involved in writing the rules, and at the end of it, what is the net false positive?
Earlier quoted context omitted.
This would make sense since Semmle would likely need access.
I'd be ready to put money on the fact that GitHub has access to all repositories, even private ones!
Earlier quoted context omitted.
First of all, Daniel Berlin is pretty senior at a reasonably large tech company a lot of us here have heard of. Secondly, I know Microsoft loves it, which is presumably where your telemetry comes from, and I know a lot of security people on Twitter are fans of the technology, but I've been asking around and "love it" is not the signal I'm getting from software security blue team people. "I installed it, I guess it do…
It’s absolutely useless in the wrong hands, so I don’t think you’d necessarily get a good signal by asking your average blue teamer. It’s a godsend for someone who spends a lot of time auditing code and has some experience writing code analysis tools. I mean think about it, if you wanted to write a query against the AST of a target, would you find that useful? Or in a given codebase, if you find one bug, would you li…
Earlier quoted context omitted.
Nothing is a panacea. Things that help move the needle without requiring tons of time or effort are useful and valuable. I'm really glad to see more efforts in this area.
While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on. (I mean this in terms of capability, not sudden popularity) It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative
I have been using Semmle daily to automate much of the vulnerability discovery process and I am extremely satisfied.
We run it over millions of lines of Java code and have not yet run into scale or perf problems.
Developing custom queries and defining security invariants in a logic language is, quite honestly, a joy.
Earlier quoted context omitted.
I've been looking quite a bit into this recently, and even though they might not be screaming it from the rooftops, Gitlab offers quite a few security-related features. There are code scanning, dependency tracking, etc. features at various levels of readiness. https://about.gitlab.com/devops-tools/ https://about.gitlab.com/stages-devops-lifecycle/secure/
They’ve had SAST tools for a few releases, but high up in the paid license types. With GitHub providing for free, they may need to move them into CE.
It just uses existing open-source analysis tools, but orchestrates them all into a single tool by coordinating a bunch of docker images.
Earlier quoted context omitted.
While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on. (I mean this in terms of capability, not sudden popularity) It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative
The Datalog part is interesting! Do they have a bunch of rules to make graph queries work nicely, like Datomic pull syntax or maybe some pattern matching syntactic sugar? Is the underlying thing still an EAVT store? Is any of that information publicly available?
https://help.semmle.com/lgtm-enterprise/user/help/generate-d... says "LGTM generates a database for each commit stored in a repository. Each database is a relational database that represents the structure of the codebase for a specific revision, or snapshot, of the code.", though a triple store could qualify as relational here. I couldn't find much more than that about the implementation details though.
"Human progress depends on the open source community." What a way to begin an article.