Live data from Hacker News

Don’t underestimate grep-based code scanning

littlemaninmyhead.wordpress.com

91–100 of 122 posts

Re: Don’t underestimate grep-based code scanning

#91

Earlier quoted context omitted.

Another thing that's not immediately obvious is the longer your search string is in grep, the faster it will find your results.

Can you explain why that's the case?

String search algorithms can cleverly skip forward when they don't find a match. They can skip forward more for longer "needle" strings.

That said grep is pretty fast period, so probably doesn't make a huge difference in practice, especially if you're IO-bound, which is common.

Re: Don’t underestimate grep-based code scanning

#92
post #72

The ability to easily grep for functions in C-like code is why I've come to appreciate projects defining their functions like: int foo_func(void) { You can grep for `^foo_func\b` to get to a declaration or definition, or `^foo_func\b.* {$` to get to a definition or `^foo_func\b.* ;` to get to a declaration. This is instead of using something like `^\w.* \bfoo_func\(`, which is what you'd need for: int foo_func(void)…

Just one more reason to love languages with trailing instead of leading types (Scala, Typescript).

    fooFunc(): Int {
"fooFunc returns an integer."

Not

"An integer is returned by fooFunc."

Re: Don’t underestimate grep-based code scanning

#93
post #91

Earlier quoted context omitted.

Can you explain why that's the case?

String search algorithms can cleverly skip forward when they don't find a match. They can skip forward more for longer "needle" strings. That said grep is pretty fast period, so probably doesn't make a huge difference in practice, especially if you're IO-bound, which is common.

The caveat being if you're Unicode aware then many of the old skipahead strategies don't work as well and have to be rolled back or disabled.

Re: Don’t underestimate grep-based code scanning

#94
post #70

Earlier quoted context omitted.

Also, check your spelling. It's a pain when items don't show up in search because of spelling issues. I've seen function names misspelled, and then every invocation just doubling down on that misspelling.

I encountered a similar problem in a c++ codebase and the debugging logs it produced. There was an error which was being reported in the logs as "weak ptr expired" or somethings like that. I grepped for it the whole source code (a gigantic project). No results. Going back and forth several times. Feeling stupid beyond imagination. Then I copy-pasted what was actually printed in the logs into my grep query (previously…

And it turns out the pointer expired on sunday morning, except in some regions where it expired on some other day.

Re: Don’t underestimate grep-based code scanning

#95

Still never going to beat AST-integrated searching like VS has for C#. Which has a regex search too.

I have stopped using the AST integrated searching vs code for Go. Instead of clicking to declaration, it is now faster in my larger code base (especially one that uses interfaces a lot) to just search for substrings. The AST search still works most of the time, but sometimes it fails and usually it is just plain slow.

Re: Don’t underestimate grep-based code scanning

#96
post #3

One thing which was not immediately obvious to me for a while: the stricter your language’s formatting is, the easier it will be to grep source code. I work a lot with Go, where all code in our repository is gofmt'ed. You can get quite far with regular expressions for finding/analyzing Go code. (And when regexps don’t cut it anymore, Go has excellent infrastructure for working with it programmatically. http://golang.…

I find that line wrapping frequently prevents me from getting complete coverage though. So it works for some cases, not for others. But I do love auto-formatters. (I'm doing web dev at the moment, so Prettier.) It is freeing to not worry about spacing, line breaks, parens, etc. All I have to do is give the computer a valid AST and it does The Right Thing.

Isn't this what the /s flag is for in your regex? Assuming you are also using /x of course.

Re: Don’t underestimate grep-based code scanning

#97

Earlier quoted context omitted.

I find that line wrapping frequently prevents me from getting complete coverage though. So it works for some cases, not for others. But I do love auto-formatters. (I'm doing web dev at the moment, so Prettier.) It is freeing to not worry about spacing, line breaks, parens, etc. All I have to do is give the computer a valid AST and it does The Right Thing.

Isn't this what the /s flag is for in your regex? Assuming you are also using /x of course.

Sort of. But there's indentation, so I need repetition.

And if I'm doing whitespace repetition, there's no great advantage in an autoformatter.

Re: Don’t underestimate grep-based code scanning

#98
post #75

Earlier quoted context omitted.

Same experience here, started with ack and switched to ag and then to rg for speed. I've found them roughly equivalent in functionality, but for those who need specific features here's a link to a feature comparison table: https://beyondgrep.com/feature-comparison/

That table is quite out of date for ripgrep, which has added a number of features. See: https://github.com/beyondgrep/website/issues/97

Thanks for ripgrep, I use it daily and was recently going through the source code to learn how to build production quality Rust apps! (https://github.com/BurntSushi/ripgrep)

Re: Don’t underestimate grep-based code scanning

#99
post #84

Earlier quoted context omitted.

They don't \0-terminate the target on overflow, so you still need to test for that condition. So most people will have a wrapper around those to ensure the \0 is there.

I think BSD has strlcpy and strlcat for exactly this reason

strlcpy has the braindamage that it returns the length of the source buffer, which means it has to traverse the entire buffer to figure out the length.

If you want to copy out the first line from a buffer that happens to be a 10TB mapped file, that strlcpy call will take a long time to finish. If you are using strncpy/strlcpy because you don't trust the src buffer is properly null terminated but you still want to stop the copy at the first null or when the buffer is full, well, you're out of luck because strlcpy is going to blast past the end of the source buffer regardless.

I would have been much happier if it had just returned a flag indicating either successful copy (0), buffer was truncated (1), or an error occurred and errno was set (-1). Possible errors could be that the src or dest was NULL or the size was 0 (ERR_BAD_ARGUMENT).

Re: Don’t underestimate grep-based code scanning

#100

Earlier quoted context omitted.

Isn't this what the /s flag is for in your regex? Assuming you are also using /x of course.

Sort of. But there's indentation, so I need repetition. And if I'm doing whitespace repetition, there's no great advantage in an autoformatter.

You do have to be more liberal with '\s*' or '\s+' instead of just ' ' in your patterns, that is true.
Post reply on HN