Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Globstar – Open-source static analysis toolkit

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Show HN: Globstar – Open-source static analysis toolkit

#11

Post by etyp »

I really love that static analyzers are pushing in this direction! I loved writing Clippy lints and I think applying that "it's just code" with custom checks is a powerful idea. I worked on a static analysis product and the rules for that were horrible, I don't blame the customers for not really wanting to write them.

Is there a general way to apply/remove/act on taint in Go checkers? I may not be digging deeply enough but it seems like the example just uses some `unsafeVars` map that is made with a magic `isUserInputSource` method. It's hard for me to immediately tell what the capabilities there are, I bet I'm missing a bit.

Re: Show HN: Globstar – Open-source static analysis toolkit

#12

Post by injuly »

etyp wrote: post #11

I really love that static analyzers are pushing in this direction! I loved writing Clippy lints and I think applying that "it's just code" with custom checks is a powerful idea. I worked on a static analysis product and the rules for that were horrible, I don't blame the customers for not really wanting to write them. Is there a general way to apply/remove/act on taint in Go checkers? I may not be digging deeply enou…

Flow analysis, especially propagation, is a hard problem to solve in the general case. IMO, the one tool that had the best, if language-specific, approach was Pyre – Facebook's type checker and static analyzer for Python.

Re: Show HN: Globstar – Open-source static analysis toolkit

#13

Post by injuly »

codepathfinder wrote: post #9

Nothing comes closer to CodeQL! If anyone is interested please checkout, codepathfinder.dev, truly opensource CodeQL alternative. Feedbacks are appreciated!

Admirable effort :)

But in its current state I don't think it actually replaces any of CodeQL's use cases. The most straight forward way to do what CodeQL does today, would to be implement a flow analysis IR (say CFG+CallGraph) on top of tree-sitter.

Even the QL grammar itself can be in tree-sitter.

Re: Show HN: Globstar – Open-source static analysis toolkit

#14

Post by [deleted] »

injuly wrote: post #13
codepathfinder wrote: post #9

Nothing comes closer to CodeQL! If anyone is interested please checkout, codepathfinder.dev, truly opensource CodeQL alternative. Feedbacks are appreciated!

Admirable effort :) But in its current state I don't think it actually replaces any of CodeQL's use cases. The most straight forward way to do what CodeQL does today, would to be implement a flow analysis IR (say CFG+CallGraph) on top of tree-sitter. Even the QL grammar itself can be in tree-sitter.

[deleted]

Re: Show HN: Globstar – Open-source static analysis toolkit

#15

Post by sanketsaurav »

markrian wrote: post #10

Interesting! Do you have a page which compares globstar against other similar tools, like Semgrep, ast-grep, Comby, etc? For instance, something like https://ast-grep.github.io/advanced/tool-comparison.html#com... .

Not at the moment, but we'll put something up soon.

We're focused on keeping globstar light-weight, so a hosted runtime is not in the roadmap (although we'll add support for running Globstar checkers natively on our commercial product DeepSource). You should be able to write any checkers in Globstar that you can write in the other tools you've listed.

Our goal is to make it very easy to write these checkers — so we'd be optimizing the runtime and our Go API for that.

Re: Show HN: Globstar – Open-source static analysis toolkit

#16

Post by codepathfinder »

injuly wrote: post #13
codepathfinder wrote: post #9

Nothing comes closer to CodeQL! If anyone is interested please checkout, codepathfinder.dev, truly opensource CodeQL alternative. Feedbacks are appreciated!

Admirable effort :) But in its current state I don't think it actually replaces any of CodeQL's use cases. The most straight forward way to do what CodeQL does today, would to be implement a flow analysis IR (say CFG+CallGraph) on top of tree-sitter. Even the QL grammar itself can be in tree-sitter.

Thanks for the feedback. That's the exact plan :raised_hands:

current state of codepathfinder is less than 5% of what codeql has implemented. As security engineer, I personally use it and i'll keep adding + closing the gap.

Feel free to contribute ideas/feedback/bugs. Super appreciable honestly!

Re: Show HN: Globstar – Open-source static analysis toolkit

#17

Post by sanketsaurav »

etyp wrote: post #11

I really love that static analyzers are pushing in this direction! I loved writing Clippy lints and I think applying that "it's just code" with custom checks is a powerful idea. I worked on a static analysis product and the rules for that were horrible, I don't blame the customers for not really wanting to write them. Is there a general way to apply/remove/act on taint in Go checkers? I may not be digging deeply enou…

Thanks! We still have a long way to go and a pretty extensive roadmap.

> Is there a general way to apply/remove/act on taint in Go checkers? I may not be digging deeply enough but it seems like the example just uses some `unsafeVars` map that is made with a magic `isUserInputSource` method. It's hard for me to immediately tell what the capabilities there are, I bet I'm missing a bit.

Assuming you're looking at the guide [1], the `isUserInputSource` is just a partial example and not a magic method (we probably should have used a better example there).

The AST for each node along with the context are exposed in the `analysis.Pass` object [2]. We don't have an example for taint analysis, but here's an example [3] of state tracking that can be used to achieve this. This is a little tedious at the moment and you'll have to do the heavy-lifting in the Go code — but this is on our roadmap to improve. We want to expose a lot more helpers to make doing things like taint analysis easily.

Here's another idea [4] we're exploring to make the YAML interface more powerful: adding support for utilities (like entropy calculation) that you can call and perform a comparison.

[1] https://globstar.dev/guides/writing-go-checker#_1-complex-pa...

[2] https://globstar.dev/reference/checker-go#analysis-function

[3] https://globstar.dev/reference/checker-go#state-tracking

[4] https://github.com/DeepSourceCorp/globstar/issues/27

Re: Show HN: Globstar – Open-source static analysis toolkit

#18

Post by [deleted] »

etyp wrote: post #11

I really love that static analyzers are pushing in this direction! I loved writing Clippy lints and I think applying that "it's just code" with custom checks is a powerful idea. I worked on a static analysis product and the rules for that were horrible, I don't blame the customers for not really wanting to write them. Is there a general way to apply/remove/act on taint in Go checkers? I may not be digging deeply enou…

[deleted]

Re: Show HN: Globstar – Open-source static analysis toolkit

#19

Post by micksmix »

This is a really interesting project!

I'd love to hear how this project differs from Bearer, which is also written in Go and based on tree-sitter? https://github.com/Bearer/bearer

Regardless, considering there is a large existing open-source collection of Semgrep rules, is there a way they can be adapted or transpiled to tree-sitter S-expressions so that they may be reused with Globstar?

Re: Show HN: Globstar – Open-source static analysis toolkit

#20

Post by sanketsaurav »

micksmix wrote: post #19

This is a really interesting project! I'd love to hear how this project differs from Bearer, which is also written in Go and based on tree-sitter? https://github.com/Bearer/bearer Regardless, considering there is a large existing open-source collection of Semgrep rules, is there a way they can be adapted or transpiled to tree-sitter S-expressions so that they may be reused with Globstar?

Thanks!

> I'd love to hear how this project differs from Bearer, which is also written in Go and based on tree-sitter? https://github.com/Bearer/bearer

The primary difference is that we're optimizing for users to write their custom rules easily. We do plan to ship built-in checkers [1] so we cover at least OWASP Top 10 across all major programming languages. We're also truly open-source using the MIT license.

> Regardless, considering there is a large existing open-source collection of Semgrep rules, is there a way they can be adapted or transpiled to tree-sitter S-expressions so that they may be reused with Globstar?

I'm pretty sure there should be a way to make that work. We believe writing checkers (and having a long list of built-in checkers) will be a commodity in a world where AI can generate S-expressions (or tree-sitter node queries in Go) for any language with very high accuracy (which is where we have an advantage as compared to tools that use a custom DSL). To that extent, we're focused on improving the runtime itself so we can support complex use cases from our YAML and Go interfaces. If the community can help us port rules from other sources to our built-in checkers, we'd love that!

[1] https://github.com/DeepSourceCorp/globstar/pulls

Post reply on HN