Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semgrep: Semantic grep for code

semgrep.dev

Post reply on HN
Print view View original

81–90 of 110 posts

Re: Semgrep: Semantic grep for code

#81

Post by wdb »

Apparently this is invalid TypeScript (cannot parse it says):

  try {
    const parsedURL = new URL(url)
    requestPath = parsedURL.pathname
  } catch (error: unknown) {
    // NOOP
  }
It's complaining about : unknown bit which one of the newer typescript eslint rules enforces.

Re: Semgrep: Semantic grep for code

#82

Post by shuringai »

underyx wrote: post #66
alpaca128 wrote: post #51

Earlier quoted context omitted.

I clicked on this thinking it's a grep that can search code snippets based on language-aware syntax matching instead of regular expressions. Agreed, this project name is misleading about what it does. The name "grep" always indicated some kind of "find a text/pattern and print results to stdout" utility. Like pgrep, which searches running processes by name and then prints their IDs.

> it's a grep that can search code snippets based on language-aware syntax matching instead of regular expressions. Hey, I'm a maintainer of Semgrep, and this sounds like a pretty good description of what the CLI can do, see this example for finding all function/class/method calls: $ semgrep -e '$NAME(...)' -l python flask_todomvc/extensions.py 4:db = SQLAlchemy() -----------------------------------------------------…

question, you mention "no more AST parsing" as a selling line for semgrep. Isn't this actually a bad thing for a staric analyzer, as ASTs are actually... abstract, and closer to the real logic of what the code does? I know analyzing text is faster but is it worth?

Re: Semgrep: Semantic grep for code

#84

Post by solumos »

thesuperbigfrog wrote: post #25

The name "Semantic Grep" does not give a good idea for what this tool is and what it does. The web page states: "Static analysis at ludicrous speed. Find bugs and enforce code standards" "grep" is short for "global regular expression print". It finds matches for the given regular expression and prints them. "Semantic Grep" is a static analyzer with configurable rules, style checks, etc. It does much more than search…

I think the name's just fine. It searches through your code in a way that you can define semantically/with more context. All of the alternatives you mention carry less meaning, and are less memorable.

This is like saying that PubNub is a bad name because it does messaging, and has little to do with pub-sub. Or hell, even that Y Combinator should be called something like StartupFactory since it's not really a recursive tool.

In short, the metaphor's close enough.

Re: Semgrep: Semantic grep for code

#85

Post by joshuamorton »

There's lots of confusion about what semgrep does here, which is kind of unfortunate. I haven't touched it much, but I have built a very similar tool (I'm one of the contributors to refex[1], which is a very similar project).

The starting point of semantic grep is very useful. When you have a big codebase, you often want to detect antipatterns, or not even antipatterns, but just uses of a thing, say you're renaming a method and want to track down the callers.

Being able to act on the AST, instead of hoping you searched up all of the variants of whitespace and line breaks and, depending on the specific example, different uses of argument passing, is really useful.

But often when you're semantically grepping, your goal is to replace something with something else (this is what refex was initially built for: to aide in large scale changes in python, as a sort of equivalent to the C++ tools that Google uses).

But then you want to shift left even further: once you have a pattern that you want to replace once, you can just enforce that a linter yell at you when anyone does it again. So it's very natural to develop a linter-style thing on top of one of these[2].

This is, as I understand it sort of the same thing that happens in C++: clang-tidy and clang-format are written on top of AST libraries that can be used for ad-hoc analysis and transformations, but you can also just plug them into a linter.

The thing is, for most organizations, enforcing code style and best practices is more valuable than apply a refactoring to 10M lines of code, because most organizations don't have 10M lines of code to refactor. That doesn't mean that these tools aren't also useful for ad-hoc transforms and exploratory analysis. They absolutely are!

[1]: https://github.com/ssbr/refex

[2]: https://github.com/ssbr/refex/tree/main/refex/fix

Re: Semgrep: Semantic grep for code

#86

Post by joj123 »

westurner wrote: post #27

Is there a more complete example of how to call semgrep from pre-commit (which gets called before every git commit) in order to prevent e.g. Python print calls (print(), print \\n(), etc.) from being checked in? https://semgrep.dev/docs/extensions/ describes how to do pre-commit. Nvm, here's semgrep's own .pre-commit-config.yml for semgrep itself: https://github.com/returntocorp/semgrep/blob/develop/.pre-co...

Out of curiosity, Is there value in doing this over (say) running a GitHub Action post commit and failing the build if it finds something nasty?

Re: Semgrep: Semantic grep for code

#87

Post by joj123 »

CGamesPlay wrote: post #83

How much does the CI service cost? I can't seem to find any information about it on the website without creating an account.

The CI service is free, with some limitations on how long the findings stay on the dashboard, SSO integration and maybe a few others. The paid version was $40/usr/mo the last time I checked Once we figured it out, it takes us a few minutes to onboard a new repo to Semgrep

Re: Semgrep: Semantic grep for code

#88

Post by Spivak »

joj123 wrote: post #86
westurner wrote: post #27

Is there a more complete example of how to call semgrep from pre-commit (which gets called before every git commit) in order to prevent e.g. Python print calls (print(), print \\n(), etc.) from being checked in? https://semgrep.dev/docs/extensions/ describes how to do pre-commit. Nvm, here's semgrep's own .pre-commit-config.yml for semgrep itself: https://github.com/returntocorp/semgrep/blob/develop/.pre-co...

Out of curiosity, Is there value in doing this over (say) running a GitHub Action post commit and failing the build if it finds something nasty?

If you can catch it before the commit is even made then why do/wait for a build?

Re: Semgrep: Semantic grep for code

#89

Post by Spivak »

wdb wrote: post #81

Apparently this is invalid TypeScript (cannot parse it says): try { const parsedURL = new URL(url) requestPath = parsedURL.pathname } catch (error: unknown) { // NOOP } It's complaining about : unknown bit which one of the newer typescript eslint rules enforces.

Apparently

    import random
    if random.randint(0,1) == 2:
        print(“hello”)
Is also unparseable.

Re: Semgrep: Semantic grep for code

#90

Post by drewdennison »

minusf wrote: post #79

probably doing something wrong but running the ci ruleset on a tiny django hobby project made all cores spin at 100% after 33% of the progress bar and made the OS almost unresponsive. ctrl-c after 5 minutes and i still had to pkill every semgrep process... never seen the M1 airbook overheat this much before.

Semgrep maintainer here. We just ordered our first M1 laptop and will debug. Thanks for the bug report
Post reply on HN