Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semgrep: Semantic grep for code

semgrep.dev

Post reply on HN
Print view View original

51–60 of 110 posts

Re: Semgrep: Semantic grep for code

#51

Post by alpaca128 »

thesuperbigfrog wrote: post #25

The name "Semantic Grep" does not give a good idea for what this tool is and what it does. The web page states: "Static analysis at ludicrous speed. Find bugs and enforce code standards" "grep" is short for "global regular expression print". It finds matches for the given regular expression and prints them. "Semantic Grep" is a static analyzer with configurable rules, style checks, etc. It does much more than search…

I clicked on this thinking it's a grep that can search code snippets based on language-aware syntax matching instead of regular expressions.

Agreed, this project name is misleading about what it does. The name "grep" always indicated some kind of "find a text/pattern and print results to stdout" utility. Like pgrep, which searches running processes by name and then prints their IDs.

Re: Semgrep: Semantic grep for code

#52

Post by aseipp »

tyingq wrote: post #42
hyper_reality wrote: post #22

This is an excellent tool to have as a security consultant, and it just keeps getting better and better. When approaching a large codebase, it enables you to write custom rules that match on certain antipatterns you've spotted that may be unique to the codebase. That's the real value of the tool, but the repository of per-language rules is also convenient for quickly finding low-hanging fruit (like every use of a pot…

I'd be careful with how much of a warm fuzzy the tool gives you. See this example from my other comment in the thread: https://news.ycombinator.com/item?id=26905880 If it were really looking at AST level data, that wouldn't have fooled it. I suspect there would be similar issues with your example of ensuring no use of eval() in PHP. So it seems okay to keep your own developers informed, but I wouldn't use it, alone,…

> If it were really looking at AST level data, that wouldn't have fooled it.

Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST. You need actual Python-specific semantic analysis that knows that all "open" functions like 'print' come from the builtins module, and thus are bound to the same identifier. They're literally built into the implementation, it's not something you can "discover" from analyzing existing Python source. Even if you had a perfectly accurate python AST it couldn't "tell" you this fact, it's a priori knowledge, and all analysis engines need a base set of facts like this that they work from.

> but I wouldn't use it, alone, to vet outside code.

I mean, nobody seems to be suggesting this though, and the OP quite literally stated the major value of the tool is enforcing domain/codebase-specific rules among a team. Which is a really good use for it! There are tons of little useful patterns you can codify this way.

Re: Semgrep: Semantic grep for code

#53

Post by tyingq »

aseipp wrote: post #52
tyingq wrote: post #42

Earlier quoted context omitted.

I'd be careful with how much of a warm fuzzy the tool gives you. See this example from my other comment in the thread: https://news.ycombinator.com/item?id=26905880 If it were really looking at AST level data, that wouldn't have fooled it. I suspect there would be similar issues with your example of ensuring no use of eval() in PHP. So it seems okay to keep your own developers informed, but I wouldn't use it, alone,…

> If it were really looking at AST level data, that wouldn't have fooled it. Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST. You need actual Python-specific semantic analysis that knows that all "open" functions like 'print' come from the builtins module, and thus are bound to the same identifier. They're literally built into the implementation, it's no…

>Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST

Perhaps I worded it poorly. Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though. So I'm curious why that skirts the rule.

>I mean, nobody seems to be suggesting this though

Not specificially, but the context is using it for security purposes with phrases like "every use of a potentially injectable function such as exec,system,etc. in PHP". Felt like that was worth commenting on.

Re: Semgrep: Semantic grep for code

#56

Post by SavantIdiot »

underyx wrote: post #19
SavantIdiot wrote: post #15

Since the capability has never existed, I don't think in terms of being able to semgrep. If that makes any sense. My brain is not wired this way, yet. Like, if you've never tasted lychee, it would never occur to you how to cook with it. I'm going to need to see some useful, real-world examples to jumpstart my brain to think this way.

Hey, I work on Semgrep. As a real world example, I just noticed today that Hashicorp uses a whole bunch of Semgrep rules on terraform-provider-aws[0]. I'd recommend reading the `message` keys to know what they intend to match, and then the `patterns` lists below to see how that's accomplished. Alternatively, we curate 1000+ community rules that you can look through as well.[1] [0]: https://github.com/hashicorp/terraf…

Nice! Thanks. This will certainly help me start to thinking in semantic grep. I can see this being an additional coverage tool and am eager to study it.

Re: Semgrep: Semantic grep for code

#57

Post by [deleted] »

tyingq wrote: post #53
aseipp wrote: post #52

Earlier quoted context omitted.

> If it were really looking at AST level data, that wouldn't have fooled it. Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST. You need actual Python-specific semantic analysis that knows that all "open" functions like 'print' come from the builtins module, and thus are bound to the same identifier. They're literally built into the implementation, it's no…

>Semgrep does look at an AST; but that counterexample is not something you can "fix" solely by looking at an AST Perhaps I worded it poorly. Dumping the python AST for builtins.print() makes it pretty clear that it's "print" though. So I'm curious why that skirts the rule. >I mean, nobody seems to be suggesting this though Not specificially, but the context is using it for security purposes with phrases like "every u…

[deleted]

Re: Semgrep: Semantic grep for code

#58

Post by robertlagrant »

smithza wrote: post #34
robertlagrant wrote: post #31

Earlier quoted context omitted.

Python you can probably do that just from the standard library. ast.parse and all that.

Perhaps, but this tool has a syntax and common interface for multiple languages. This is a huge resource for IT/security. I could imagine this being a helpful refactoring tool as well for CLI.

No need to overstate. It's fine, but the rules seem to need to be very bespoke, and any language's parser will do a better job of figuring out syntax.

Re: Semgrep: Semantic grep for code

#59

Post by avodonosov »

hn_throwaway_99 wrote: post #39

I currently use a highly opinionated ESLint config (based on the airbnb one) together with strict checking in my TypeScript config, and it is configured to run on every commit with husky git hooks. The example given on the Semgrep homepage is an exact match to one that exists in my ESLint config (eslint's no-console rule). How does Semgrep compare to ESLint+a strict tsconfig?

How do you deal with false positives?

If the commit hook rejects anything where rules are triggered, a way to force the comrit is needed for the cases when the rule finding is not reanny an issue.

Upd: I found that the --no-verify option can be used in many cases

Re: Semgrep: Semantic grep for code

#60

Post by IshKebab »

prepend wrote: post #50
IshKebab wrote: post #40

Earlier quoted context omitted.

Kind of crazy that you can make a tool like this in the modern age that isn't cross platform. Maybe they just can't face the Python packaging nightmare on Windows. Also kind of surprising it's written in Python given that they advertise its speed.

I’m always surprised at stuff I take for granted that doesn’t work on Windows. So yeah, it seems like cross-platform should be easy but since my dev environment is zsh, it’s easy for my stuff to work sort of “everywhere but Windows.” Add to that that the reason things fail on Windows is usually something Windows specific and “their fault.” So it’s unusual for me to fire up a Windows VM just to sanity check my code. A…

This is an open source project on Github. They can use Github Actions which has free runners for Windows, Mac and Linux.
Post reply on HN