Live data from Hacker News

Welcoming Semmle to GitHub

github.blog

51–60 of 110 posts

Re: Welcoming Semmle to GitHub

#51

Earlier quoted context omitted.

It's being run frequently across the entire Windows OS repo. I have heard there is more work to be done to make it scale better, but it can scale.

Am Microsoft. Mountains were moved to make it scale, but that has been achieved. Semmle can scale with work - it just takes a lot of effort and code.

What had to be done to make it scale?

Re: Welcoming Semmle to GitHub

#52

Earlier quoted context omitted.

The funny thing is I was working on a pitch to get Atlassian to buy them so they don't end up in Microsoft's hands. I thought integration with a repo company would be good since they could cross-sell it for code understanding and maintenance. Then I see this article. (sighs) At least I got it right on the type of company that would grab them. I'd push something else for security, though, to complement it. RV-Match is…

Why would Atlassian be better?

They were kind of a default since there's only a few huge ones. Main win: it's not Microsoft. MS already has lots of internal tools from MS Research they were wasting. They probably couldve built Semmle themselves. They're also a known patent troll. I don't know if Semmle's methods are patented, though.

I'd rather a company like them not acquire them in favor of one that is constantly developing new services with no attempts to financially drain 3rd parties. Not to say good won't come out of Github integration given huge number of projects in it.

Re: Welcoming Semmle to GitHub

#53
post #5

Earlier quoted context omitted.

Nothing is a panacea. Things that help move the needle without requiring tons of time or effort are useful and valuable. I'm really glad to see more efforts in this area.

While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on. (I mean this in terms of capability, not sudden popularity) It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative

I would love to hear more about this data, as everyone I know who has used it for vulnerability finding has very good things to say. Semmle have also demonstrated its capabilities with some high profile examples.

Re: Welcoming Semmle to GitHub

#54
post #4

Earlier quoted context omitted.

Semmle is basically datalog over source code. For what it works for, it works nice. But it is not a pancaea. Security vulnerability finding is almost certainly the wrong target for Semmle - I am unsure why they are trying to push that angle. There are much better stories in things like refactoring and understanding. (I say this having overseen a number of deployments for various reasons, some successful, some not)

You’re wrong on that, security teams at the major tech companies love it, especially for variant analysis. Ask your coworkers at Google! One of which recently left to become Semmle’s Chief Security Officer.

First of all, Daniel Berlin is pretty senior at a reasonably large tech company a lot of us here have heard of.

Secondly, I know Microsoft loves it, which is presumably where your telemetry comes from, and I know a lot of security people on Twitter are fans of the technology, but I've been asking around and "love it" is not the signal I'm getting from software security blue team people. "I installed it, I guess it does some stuff, we never think about it" is the modal feedback I've seen.

I'm very interested in hearing success stories about this; the problem Semmle addresses is a huge part of the cost basis for my practice, and I'd love to hear that someone has gotten it working well.

Re: Welcoming Semmle to GitHub

#55

Earlier quoted context omitted.

While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on. (I mean this in terms of capability, not sudden popularity) It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative

I would love to hear more about this data, as everyone I know who has used it for vulnerability finding has very good things to say. Semmle have also demonstrated its capabilities with some high profile examples.

Who have you talked to about it? Outside Mozilla and Microsoft?

Re: Welcoming Semmle to GitHub

#56

The linked blog post [0] and the new security marketing page [1] both have a little more detail on what this actually means. Basically, Semmle offers a static analysis tool that operates on your source code as a graph (from what I understand) and points out bugs and security holes in your code. Github is now offering that for free on repos at all tiers. [0] https://github.blog/2019-09-18-securing-software-together/ […

The funny thing is I was working on a pitch to get Atlassian to buy them so they don't end up in Microsoft's hands. I thought integration with a repo company would be good since they could cross-sell it for code understanding and maintenance. Then I see this article. (sighs) At least I got it right on the type of company that would grab them. I'd push something else for security, though, to complement it. RV-Match is…

Just a small fraction of the industry's ongoing software development is done in C. Obvious, Google, Microsoft, Apple, and Mozilla still write a lot of it, but you don't acquire a whole company to address 4-and-change customers.

Re: Welcoming Semmle to GitHub

#57
post #4

Earlier quoted context omitted.

Semmle is basically datalog over source code. For what it works for, it works nice. But it is not a pancaea. Security vulnerability finding is almost certainly the wrong target for Semmle - I am unsure why they are trying to push that angle. There are much better stories in things like refactoring and understanding. (I say this having overseen a number of deployments for various reasons, some successful, some not)

I've seen coworkers run semmle queries across the entire Windows OS codebase and find hundreds of issues which were/could result in security vulnerabilities. They've also leveraged it for variant analysis. If I'm not mistaken, the security teams are the largest internal users of Semmle at Microsoft. You're right though, it's not a panacea, and it could probably be great for other uses too.

Any idea what the false positive rate is?

Re: Welcoming Semmle to GitHub

#58
post #56

Earlier quoted context omitted.

The funny thing is I was working on a pitch to get Atlassian to buy them so they don't end up in Microsoft's hands. I thought integration with a repo company would be good since they could cross-sell it for code understanding and maintenance. Then I see this article. (sighs) At least I got it right on the type of company that would grab them. I'd push something else for security, though, to complement it. RV-Match is…

Just a small fraction of the industry's ongoing software development is done in C. Obvious, Google, Microsoft, Apple, and Mozilla still write a lot of it, but you don't acquire a whole company to address 4-and-change customers.

They'd have them retarget to what's popular post-acquisition in my concept. Especially among their paying customers. Keep adding languages or just useful things for it to look for.

Re: Welcoming Semmle to GitHub

#59
post #56

Earlier quoted context omitted.

Just a small fraction of the industry's ongoing software development is done in C. Obvious, Google, Microsoft, Apple, and Mozilla still write a lot of it, but you don't acquire a whole company to address 4-and-change customers.

They'd have them retarget to what's popular post-acquisition in my concept. Especially among their paying customers. Keep adding languages or just useful things for it to look for.

Security problems aren't interchangeable between languages, and C has a very particular set of concerns that don't translate.
Post reply on HN