The CI use case is cool, and probably makes more money. But I would really love to see a CLI for optimized search and replace. It seems that they have search available on the CLI however I can't see any replace. And most of the options are focused on running the rule config instead of adhoc replacements.
Introducing Semgrep and r2c
11–20 of 23 posts
Re: Introducing Semgrep and r2c
#12Earlier quoted context omitted.
First of all, I love the idea of semgrep, but can't use it since we're using C++. Is there any chance for C++ support in the future?
The good news is that we’ve replaced almost all the homegrown parsers that were written while the tool was at Facebook and we’re using the now tree-sitter project, which already has parsers for 40+ languages. There is a tree-sitter-cpp project we can and will eventually integrate! The bad news is this requires the code to not use heavily macros to be parseable as-is. So really the difficulty is not C++ but rather the…
I guess you could run the preprocessor and then run tree-sitter-cpp / semgrep on the preprocessed output, but the problem would then be trying to tie any findings from that to the original source.
Do gcc/clang/any other preprocessor create "source maps" that could facilitate that? GCC looks like it has a `-fdebug-cpp` that "[...] dumps debugging information about location maps. Every token in the output is preceded by the dump of the map its location belongs to."
Re: Introducing Semgrep and r2c
#13Hey HN, I’m the author of this post and a contributor to Semgrep. Happy to answer questions and hear feedback! I’m excited to try to lower the barrier to writing a simple lint (or more complex program analysis) that previously only a static analysis expert could do; we’ve gotten contributions from people who don’t know what an abstract syntax tree is! The userbase for Semgrep is almost evenly split between security e…
Is Semmle, offering CodeQL language and LGTM service, and recently acquired by Github, doing a similar thing (https://semmle.com/)? If so, how does Semgrep compare to CodeQL?
Edit: There is a help entry: https://semgrep.dev/docs/faq/#how-is-semgrep-different-from-...
Re: Introducing Semgrep and r2c
#14Interesting. You can try it out here: https://semgrep.dev/editor/
It doesn't appear to catch the following when searching for exec(...) in the following python code:
not_exec = exec
not_exec('rm -rf /')
Re: Introducing Semgrep and r2c
#15Earlier quoted context omitted.
The good news is that we’ve replaced almost all the homegrown parsers that were written while the tool was at Facebook and we’re using the now tree-sitter project, which already has parsers for 40+ languages. There is a tree-sitter-cpp project we can and will eventually integrate! The bad news is this requires the code to not use heavily macros to be parseable as-is. So really the difficulty is not C++ but rather the…
I guess you could run the preprocessor and then run tree-sitter-cpp / semgrep on the preprocessed output, but the problem would then be trying to tie any findings from that to the original source. Do gcc/clang/any other preprocessor create "source maps" that could facilitate that? GCC looks like it has a `-fdebug-cpp` that "[...] dumps debugging information about location maps. Every token in the output is preceded b…
The preprocessed output of GCC and Clang usually contains the file names.
Re: Introducing Semgrep and r2c
#16Interesting. You can try it out here: https://semgrep.dev/editor/ It doesn't appear to catch the following when searching for exec(...) in the following python code: not_exec = exec not_exec('rm -rf /') Edited to include language
[deleted]
Re: Introducing Semgrep and r2c
#17Earlier quoted context omitted.
The good news is that we’ve replaced almost all the homegrown parsers that were written while the tool was at Facebook and we’re using the now tree-sitter project, which already has parsers for 40+ languages. There is a tree-sitter-cpp project we can and will eventually integrate! The bad news is this requires the code to not use heavily macros to be parseable as-is. So really the difficulty is not C++ but rather the…
I guess you could run the preprocessor and then run tree-sitter-cpp / semgrep on the preprocessed output, but the problem would then be trying to tie any findings from that to the original source. Do gcc/clang/any other preprocessor create "source maps" that could facilitate that? GCC looks like it has a `-fdebug-cpp` that "[...] dumps debugging information about location maps. Every token in the output is preceded b…
Right; but this turns out to be pretty tricky in practice. I've attempted to do this for even relatively straightforward code (libsodium--complex in implementation, though not in API) with libclang and it was not particularly pleasant.
Some prior art for reference: https://github.com/bytedeco/javacpp/issues/51
Re: Introducing Semgrep and r2c
#18The CI use case is cool, and probably makes more money. But I would really love to see a CLI for optimized search and replace. It seems that they have search available on the CLI however I can't see any replace. And most of the options are focused on running the rule config instead of adhoc replacements.
The CLI does have an --autofix flag, but the replacement it uses has to be specified through a local config file rather than as a command line arg. There is a ticket that though!
https://github.com/returntocorp/semgrep/issues/840
Here are docs for what exists currently https://semgrep.dev/docs/experiments/#autofix
Re: Introducing Semgrep and r2c
#19Interesting. You can try it out here: https://semgrep.dev/editor/ It doesn't appear to catch the following when searching for exec(...) in the following python code: not_exec = exec not_exec('rm -rf /') Edited to include language
Good catch. Currently we only support constant propagation for literals. Here's a working example:
$ semgrep -e "not_exec('somestr)"
foo = "somestr"
not_exec(foo)
In your example, we don't propagate exec because it's not seen as a literal -- that's a TODO for sure. See https://github.com/returntocorp/semgrep/issues/1645 for a longer discussion!
Re: Introducing Semgrep and r2c
#20Earlier quoted context omitted.
I guess you could run the preprocessor and then run tree-sitter-cpp / semgrep on the preprocessed output, but the problem would then be trying to tie any findings from that to the original source. Do gcc/clang/any other preprocessor create "source maps" that could facilitate that? GCC looks like it has a `-fdebug-cpp` that "[...] dumps debugging information about location maps. Every token in the output is preceded b…
The preprocessed output of GCC and Clang usually contains the file names.
Better than that, compilers can be told to include line numbers too.
This thread would love to learn about Compiler Explorer https://gcc.godbolt.org/ which works for C++ and many other languages.