Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don’t underestimate grep-based code scanning

littlemaninmyhead.wordpress.com

Post reply on HN
Print view View original

41–50 of 122 posts

Re: Don’t underestimate grep-based code scanning

#41

Post by burntsushi »

Annatar wrote: post #14
K0nserv wrote: post #2

Just a small note that I would highgly recommend ripgrep[0] over standard grep. It's another modern tool that has been created by leveraging Rust and it's from BurntSushi[1] who is excellent. 0: https://github.com/BurntSushi/ripgrep 1. https://github.com/BurntSushi

This program re-invented find + egrep. It is written in Rust which isn't available on anything other than Linux, OS X and Windows. I'll just stick to /usr/bin/find and /usr/bin/egrep programs. That works just fine without having to port Rust before trying to build a re-invention of find and egrep.

> It is written in Rust which isn't available on anything other than Linux, OS X and Windows.

That's not true. As ripgrep's README says, it's available on OpenBSD, FreeBSD and NetBSD.

Re: Don’t underestimate grep-based code scanning

#42

Post by sxldier »

pjc50 wrote: post #29
Annatar wrote: post #22

Earlier quoted context omitted.

Real fgrep does not implement -r because that would be implementing tools within tools, which is against the UNIX®️ philosophy. Try /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' and run it several times so that the filesystem cache is primed.

.. which falls over as soon as you have a file with a space in the name. Edit: this highlights the big weakness in the "UNIX philosophy", in which the only record delimiter that's conventionally recognized in pipelines is the newline but the shell recognizes characters as filename delimiters that are also allowed in filenames . Causing a cascade of delimiter bugs. Sometimes you really do need a bit more structure to…

It's easy to unsafely handle filenames. I've seen it at my job where we do a lot of bash scripting. However, there are good guides on doing it the right way. [0]

[0] https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_....

Re: Don’t underestimate grep-based code scanning

#43

Post by yjftsjthsd-h »

robohoe wrote: post #36
adwn wrote: post #27

Earlier quoted context omitted.

> Real fgrep does not implement -r That's BS. The fgrep on my system – GNU grep 3.1 – provides recursive search (-r). What now, are you claiming that's not "real fgrep"? [1] > that would be implementing tools within tools, which is against the UNIX®️ philosophy Even more BS. Or are you telling me that "rm -r" is also against the "UNIX philosophy"? > /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/f…

This is why Unix is great. It gives you enough tools to shoot yourself in the foot.

"Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things."

Re: Don’t underestimate grep-based code scanning

#44

Post by yjftsjthsd-h »

Annatar wrote: post #17
adrianN wrote: post #15

Earlier quoted context omitted.

It "reinvented" it and made it dramatically faster. Rust is also available on FreeBSD. I don't think you actually need Rust to run ripgrep. It's not like it's an interpreted language.

Since when do FreeBSD executables run on the illumos family of operating systems, as far as I know, there is no freebsd-branded zone yet in illumos? Dramatically faster? Is there any scientific evidence to that? If it were me, I wouldn't rush to make assumptions.

> Since when do FreeBSD executables run on the illumos family of operating systems, as far as I know, there is no freebsd-branded zone yet in illumos?

Moving goalposts.

> Dramatically faster? Is there any scientific evidence to that?

Yes, although the difference is significant enough that most people don't bother.

> If it were me, I wouldn't rush to make assumptions.

You literally are rushing to make assumptions, it's just that you're making assumptions against ripgrep.

Re: Don’t underestimate grep-based code scanning

#45

Post by [deleted] »

pepper_sauce wrote: post #5
K0nserv wrote: post #2

Just a small note that I would highgly recommend ripgrep[0] over standard grep. It's another modern tool that has been created by leveraging Rust and it's from BurntSushi[1] who is excellent. 0: https://github.com/BurntSushi/ripgrep 1. https://github.com/BurntSushi

Why is it better than grep?

[deleted]

Re: Don’t underestimate grep-based code scanning

#46

Post by masklinn »

Annatar wrote: post #13
Canada wrote: post #6

Earlier quoted context omitted.

It's way faster, which is great when you're working with big repos. It's designed for recursively searching through a lot of files.

However fast it is, it's going to have a tough time beating /usr/bin/fgrep.

Both grep and rg will do literal search with all the whizbang optimisations they could think of if they're given a literal (a string with no metacharacters) or the -F option (to not interpret the input as a regex).

Which you'd know if you'd wondered, because that's something /u/burntsushi regularly explains: https://blog.burntsushi.net/ripgrep/#literal-optimizations

Re: Don’t underestimate grep-based code scanning

#47

Post by anon1253 »

I tend to work a lot in Lisp and XML, both are more or less trees if you squint (with the Lisp syntax famously being the AST due to homoiconicity) and it always makes me wonder if there are better command line tree search or tree diff algorithms out there (extra awesome if it works with git merge strategies). I mean whitespace preference is fine and all, but sometimes you just don’t care :p

Re: Don’t underestimate grep-based code scanning

#49

Post by tannhaeuser »

The post's core message seems to be lost on HN. It's about screening sources for supposedly insecure and/or injection-prone funcs using simple text scanning (such as strcat, which however is considered in iOS apps when it is a C std API func); supposedly grepability is also about quickly finding code locations of messages and variables. But comments are all about Rust or Go superiority, irrelevant grep implementation details, and AST-based code analysis tools when these are specifically dismissed in TFA as producing too many false positives. Talk about bubbles and echo chambers.

Re: Don’t underestimate grep-based code scanning

#50

Post by felixfbecker »

pjc50 wrote: post #29
Annatar wrote: post #22

Earlier quoted context omitted.

Real fgrep does not implement -r because that would be implementing tools within tools, which is against the UNIX®️ philosophy. Try /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' and run it several times so that the filesystem cache is primed.

.. which falls over as soon as you have a file with a space in the name. Edit: this highlights the big weakness in the "UNIX philosophy", in which the only record delimiter that's conventionally recognized in pipelines is the newline but the shell recognizes characters as filename delimiters that are also allowed in filenames . Causing a cascade of delimiter bugs. Sometimes you really do need a bit more structure to…

And that’s why PowerShell is awesome :)
Post reply on HN