Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don’t underestimate grep-based code scanning

littlemaninmyhead.wordpress.com

Post reply on HN
Print view View original

111–120 of 122 posts

Re: Don’t underestimate grep-based code scanning

#111

Post by burntsushi »

Annatar wrote: post #110
sctb wrote: post #61

Earlier quoted context omitted.

Could you please review the guidelines and post less rudely and antagonistically? https://news.ycombinator.com/newsguidelines.html

When they post these misguided reinventions of the wheel which on top of that require gargantuan effort to get working for something as trivial as a find + grep, they antagonize me; and then they argue about GNU tools having never seen or worked on a real UNIX, it's blind leading the blind. Infuriating.

Except tools like ripgrep aren't equivalent to find + grep. There is no simple invocation of find + grep that does what ripgrep does automatically. `git grep` would be closer.

You talk about being antagonized, but many of your comments in this thread have stated either outright incorrect things, or moved the goalposts, without acknowledging either one even when others point it out. Talk about infuriating.

Just because a misguided person such as myself wrote a piece of software doesn't mean you get to be rude to everyone who talks about it or suggests it.

Re: Don’t underestimate grep-based code scanning

#112

Post by johnny-lee »

I've gone down this road years ago.

While there's no install and initial results are quick to appear, the false positives that grep or any string search tool generates will make the cynics shoot down this simple attempt to find problems in the source code.

Problems that arose:

- what about use of those questionable APIs/constants in strings (perhaps for logging) or in comments?

- some of the APIs listed in the article were only questionable when certain values were used - sometimes you can get grep/search tool of choice to play along, but if the API call spans multiple lines or the constant has been assigned to a variable that is used instead, then a plain string search won't help.

- it's hard to ignore previously flagged but accepted uses of the API/constants.

- so there's a possible bug reported, but devs usually want to see the context of the problem (the code that contains the problem) quickly/easily. Some text editors can grok the grep output and place the cursor at the particular line/character with the problem, some can't.

If you go down that road to try and reduce false positives, you'll end up with a parser for your development language of choice.

Re: Don’t underestimate grep-based code scanning

#113

Post by KuhlMensch »

I do a few VERY SIMPLE greps. The most useful, is a pre-commit hook to check no blacklisted env vars exist in the commit diff. So, useful.

Grepping leans-in to shell. Though if you have other environments available (python, javascript etc), it makes sense to lean-into them e.g I use JavaScript examine my package.json to ensure my dependency SemVers' are "exact".

That said, I rarely write static-analysis scripts: In JavaScript-world there is already a plethora of easily configurable linting & type-checking tools. If I wanted to focus in on static-analysis etc I'd probably reach for https://danger.systems/js/

SideNote: My CI generates a metrics.csv file, which serves as a "metric catch-all" for any script I might write e.g. grep to count "// TODO" and "test.skip" strings, plus my JavasScript tests generate performance metrics (via monkey-patching React).

I don't actually DO ANYTHING with these metrics, but I'm quite happy knowing the CI is chugging away at its little metric diary. One day I'll plug it into something.

Re: Don’t underestimate grep-based code scanning

#114

Post by peterstensmyr »

splittingTimes wrote: post #86
felipeerias wrote: post #60

Earlier quoted context omitted.

Related to this, it is generally a very good idea to be strict when naming functions, parameters, variables, etc. so that each concept has exactly one name throughout the codebase.

But how do you effectively orhanize/enforce this for a code base of several million LOC where geographically distributed teams are working on different ends of the system all the time? The amount of cross team coordination is staggering.

Fail verification in CI if the change doesn’t pass your checks. You can check anything, such as whether it has a duplicate name already in the codebase.

Re: Don’t underestimate grep-based code scanning

#115

Post by dmit »

burntsushi wrote: post #111
Annatar wrote: post #110

Earlier quoted context omitted.

When they post these misguided reinventions of the wheel which on top of that require gargantuan effort to get working for something as trivial as a find + grep, they antagonize me; and then they argue about GNU tools having never seen or worked on a real UNIX, it's blind leading the blind. Infuriating.

Except tools like ripgrep aren't equivalent to find + grep. There is no simple invocation of find + grep that does what ripgrep does automatically. `git grep` would be closer. You talk about being antagonized, but many of your comments in this thread have stated either outright incorrect things, or moved the goalposts, without acknowledging either one even when others point it out. Talk about infuriating. Just becaus…

Sorry, replying to you while responding to the parent because their post is already flagged.

Annatar, both the Rust compiler and ripgrep are available as packages in pkgsrc. The number of hoops one needs to jump in order to use this tool on your niche platform is exactly one. And that hoop is not even on fire.

Keep moving those goal posts though. Hopefully you can move them far enough to keep the Venn diagram of your mistruths and people who recognize them completely disjoint.

Re: Don’t underestimate grep-based code scanning

#116

Post by JetSpiegel »

burntsushi wrote: post #109
jandrese wrote: post #102

Earlier quoted context omitted.

I noticed severe slowdowns when passing in the /u flag on my regexes, even with big fixed ASCII strings in the middle of the patterns. They were taking 10 times as long to complete.

That doesn't imply that things like Boyer-Moore suddenly stop being effective. Without more details (which regex engine? what regex? what corpus? which programming language?) it's impossible to state the cause, but it could be as simple as the regex engine not being smart enough to use a literal searcher in that case.

Since you don't want to toot your own horn, and you saved me many hours of not waiting for searches, I'll link to ripgrep here

https://github.com/BurntSushi/ripgrep

Re: Don’t underestimate grep-based code scanning

#117

Post by JetSpiegel »

valbaca wrote: post #74
fredley wrote: post #52

Don't use grep. Use ag[0], which is specifically designed for searching code. It's much faster, honors .gitignore, and the output can be piped back through grep if you like. ag FooBar | grep -v Baz It's in brew/apt/yum etc as `the_silver_searcher` (although brew install ag works fine too). 0: https://github.com/ggreer/the_silver_searcher

+1 Here's why: ag provides sane default and settings for developers. grep is ubiquitous and great, but to do what most developers want it requires some guidance, whereas ag focuses on being what you want most of the time. What I mean by that is that I enjoy the smart-case sensitivity (as in, if there are not caps in my pattern, then it defaults to case-insensitive but if I have any caps in my pattern, it uses case-se…

ripgrep > > > ag

Re: Don’t underestimate grep-based code scanning

#118

Post by time0ut »

johnny-lee wrote: post #112

I've gone down this road years ago. While there's no install and initial results are quick to appear, the false positives that grep or any string search tool generates will make the cynics shoot down this simple attempt to find problems in the source code. Problems that arose: - what about use of those questionable APIs/constants in strings (perhaps for logging) or in comments? - some of the APIs listed in the articl…

I haven't tried this approach, but having spent years using one of the best commercial SAST tools, I'm reluctant to dismiss it too quickly.

My SAST generates tons of false positives and is unforgivably slow. If this is orders of magnitude faster, it might be worth the extra false positives.

As a side note, my dream is a SAST that comments directly in the PR like a human reviewer would. Maybe that exists?

Re: Don’t underestimate grep-based code scanning

#119

Post by johnny-lee »

time0ut wrote: post #118
johnny-lee wrote: post #112

I've gone down this road years ago. While there's no install and initial results are quick to appear, the false positives that grep or any string search tool generates will make the cynics shoot down this simple attempt to find problems in the source code. Problems that arose: - what about use of those questionable APIs/constants in strings (perhaps for logging) or in comments? - some of the APIs listed in the articl…

I haven't tried this approach, but having spent years using one of the best commercial SAST tools, I'm reluctant to dismiss it too quickly. My SAST generates tons of false positives and is unforgivably slow. If this is orders of magnitude faster, it might be worth the extra false positives. As a side note, my dream is a SAST that comments directly in the PR like a human reviewer would. Maybe that exists?

The SAST program is probably doing a lot more than a string search tool does.

If the SAST has to process C/C++ source code, then the SAST will parse all the #include'd header files. The SAST may track values to determine if illegal/uninitialized values are used.

A string search tool will skip doing all of that.

If the class of problems you're looking for contains only bad functions/constants, then a string search tool may be fine.

But as I mentioned before, the string search tool may get confused if these bad strings occur in strings/comments/irrelevant #if/#else/#elif sections.

There are another class of bugs dealing with data values which a string search tool can't deal with easily.

As an example, PC-Lint lists the type of problems the program may flag - https://www.gimpel.com/html/lintchks.htm. A string search tool won't know about classes and virtual destructors or other concepts relevant to the programming language in question.

For the string search tool, you'd either invoke the search string tool several times with different search strings for the same source code or slightly more efficient, have one long search string containing all your search strings as alternate search targets for the string search tool.

Either case, when the string search tool spits out a positive result, it won't explain why there is a problem. The dev will have to know or lookup the problem associated with that search result.

When I worked on this area, C/C++ compilers stopped at syntax errors. Most have gotten better at flagging popular problems like variable assignments within if statements, operator precedence bugs, and printf-format string bugs.

Some divisions at Microsoft required devs to run a lightweight SAST before committing changes to locate possible problems ASAP.

It's relatively easy to integrate an SAST into your build system to scan the modified source code just before you're ready to commit the changes.

Re: Don’t underestimate grep-based code scanning

#120

Post by splittingTimes »

peterstensmyr wrote: post #114
splittingTimes wrote: post #86

Earlier quoted context omitted.

But how do you effectively orhanize/enforce this for a code base of several million LOC where geographically distributed teams are working on different ends of the system all the time? The amount of cross team coordination is staggering.

Fail verification in CI if the change doesn’t pass your checks. You can check anything, such as whether it has a duplicate name already in the codebase.

I would be interested to know, which CI tool can check "that each concept has exactly one name throughout the codebase."

I thought code reviews are the only way and then you need to have every Dev aligned and on the same page on this topic... Which never happens. :/

Post reply on HN