Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ripgrep – A new command line search tool

blog.burntsushi.net

Post reply on HN
Print view View original

141–150 of 219 posts

Re: Ripgrep – A new command line search tool

#141

Post by firebones »

saurik wrote: post #125
pcwalton wrote: post #65

Earlier quoted context omitted.

Nobody is under the slightest bit of moral obligation to work with existing projects when starting a new one. There is nothing wrong with competition.

I'm trying to describe just how disappointed I am in hearing you say that, but the words just aren't coming to me today.

If you invert the two elements of pcwalton's comment, another way of putting it that might sound less disappointing to your ear would be this: competition is another way of contributing to other projects, if done openly, collaboratively, and in a constructive way.

One of the delightful aspects of this entire HN thread is the interchange between the authors of rg and ag with regard to burntsushi's detailed explanation of the tradeoffs involved, and how the ag author considers making some changes based on learning and philosophy detailed by burntsushi. This interchange is a great bazaar moment in the sense that new ground was staked out. If the proposed moral obligation would be to simply contribute PRs to existing projects (for a slew of philosophical architectural and features differences like Unicode support, etc.), then innovation slows to a crawl because of passivity constraints, bikeshedding, etc.

If you have the talent and capacity to build a better alternative, and share it openly, and explain it so well, as burntsushi has in this domain, then the only moral imperative I see is that you should do so.

Re: Ripgrep – A new command line search tool

#142

Post by wyldfire »

ggreer wrote: post #103
burntsushi wrote: post #66

Earlier quoted context omitted.

Thanks for the response! Some notes: 1. In my benchmarks, I do control for line numbers by either explicitly making it a variable (i.e., when you see `(lines)`) or by making all tools count lines to make the comparison fair. For the most part, this only tends to matter in the single-file benchmarks. 2. For memory maps, you might get very different results depending on your environment. For example, I enabled memory m…

In terms of core features, ripgrep is totally there. It searches fast . It ignores files pretty accurately. It outputs results in a pleasant and useful format. If a new user tries rg, they'll be very happy. My warning about the feature differences was meant to temper ag users' expectations. There are lots of little things that ag users are accustomed to that are either different or missing in ripgrep. Off the top of…

> detects stdout redirects

Is there a portable way for you to find out that stdout is connected to output.txt? isatty() only tells you that there may be a redirection and I suppose on linux you could use /proc/self/fd/1 but I don't know how to do it portably.

Re: Ripgrep – A new command line search tool

#143

Post by Manishearth »

saurik wrote: post #129
burntsushi wrote: post #78

Earlier quoted context omitted.

> (or maybe had some core in there that was, while leaving the other UI parts he wants to change on top) But I did! Have you looked at the dependency list of ripgrep? It's utterly filled with tons of tools that you can pick out and use in other projects for any purpose you like. I didn't mention this in the blog post because there's already too much there, but sure, here they are: memchr - Fast single byte search: ht…

> I'm sure five years from now, when I've moved on to other problems, someone will come along and beat ripgrep, and I can only hope that they write about it. :-) I contend that the world would be a much better place if instead of someone building a new tool which beats yours, they worked to improve your tool (which, at the point where you moved on, would hopefully have been granted to a body of separate maintainers,…

> Providing some of the parts,

He just gave a laundry list of parts which grep or any other tool could use, and gives a thorough explanation of what benefits each one gets you in the blog post.

He's also having a seemingly productive discussion with the ag author on HN, which could result in ag sharing components.

Ripgrep seems to be a fun side project for the author. Making it posix compatible and getting involved in the politics of changing the language and algorithms of a major tool might not be considered fun by the author. This is totally reasonable. Especially given that Rust isn't really the reason why rg is fast -- justifying Rust is harder in this case. But if you like Rust (and not C/++), then it's even less fun to work on a C/++ codebase.

The blog post is written in a way that any tool can pick up the same perf improvements if it wishes (often by just picking up a dependency). The author has been collaborative, not combative, with the ag author here (and presumably with other tool maintainers if they come forth). This is very much in the spirit of open source, since even if things superficially compete they still can share code and ideas. You're being very unreasonable here.

Re: Ripgrep – A new command line search tool

#144

Post by bpchaps »

burntsushi wrote: post #83
krylon wrote: post #42

When I use grep (which is fairly regularly), the bottleneck is nearly always the disk or the network (in case of NFS/SMB volumes). Just out of curiosity, what kind of use case makes grep and prospective replacements scream? The most "hardcore" I got with grep was digging through a few gigabytes of ShamePoint logs looking for those correlation IDs, and that apparently was completely I/O-bound, the CPUs on that machine…

> Just out of curiosity, what kind of use case makes grep and prospective replacements scream? Unicode? Check out the subtitle benchmarks in the blog post. In the best case, grep is a little slower. In the worst case, grep is orders of magnitude slower. ripgrep achieves speed by building UTF-8 decoding straight into its DFA regex engine (well, strictly speaking, this is Rust's regex engine, not ripgrep). The other ca…

Maybe I'm unique to this, but that sort of default would drive me batshit insane. Not all of us are programmers by trade who use git. It's just another gotcha to keep track of. I'd seriously recommend removing that as a default.

That said, this is an overall awesome project.

Re: Ripgrep – A new command line search tool

#145

Post by dgr-nn-glth »

livestockboy wrote: post #92
CJefferson wrote: post #24

Earlier quoted context omitted.

Well in the interests of making contraversal statements, that's now two Rust programs I plan on using day-to-day, and zero for Haskell, Scala, and I'm sure a few other languages I'm not remembering.

Pandoc Xmonad Darcs Git-annex Although, it would be very hard to write SIMD performant code in Haskell, even its string libraries don't use any optimization tricks.

I was excited when I heard of it, but I thought Darcs had run into some major issues with patch theory and exponential merges. Other than it being written in Haskell, why would I use it over git?

Re: Ripgrep – A new command line search tool

#146

Post by burntsushi »

ggreer wrote: post #139
burntsushi wrote: post #135

Earlier quoted context omitted.

Let's do it. https://github.com/BurntSushi/ripgrep/issues/40

…and merged: https://github.com/ggreer/the_silver_searcher/pull/974 I'll tag a new release in a day or two. Also, it looks like the sift author is getting on the .ignore train: https://github.com/svent/sift/issues/78#issuecomment-2493352... This worked out pretty well. :)

Same! https://github.com/BurntSushi/ripgrep/pull/41

Agreed :-)

Re: Ripgrep – A new command line search tool

#147

Post by burntsushi »

bpchaps wrote: post #144
burntsushi wrote: post #83

Earlier quoted context omitted.

> Just out of curiosity, what kind of use case makes grep and prospective replacements scream? Unicode? Check out the subtitle benchmarks in the blog post. In the best case, grep is a little slower. In the worst case, grep is orders of magnitude slower. ripgrep achieves speed by building UTF-8 decoding straight into its DFA regex engine (well, strictly speaking, this is Rust's regex engine, not ripgrep). The other ca…

Maybe I'm unique to this, but that sort of default would drive me batshit insane. Not all of us are programmers by trade who use git. It's just another gotcha to keep track of. I'd seriously recommend removing that as a default. That said, this is an overall awesome project.

The default isn't going to change. The Silver Searcher has, IMO, proven that it's a good default. I've spoken with so many people that love it, myself included.

If you want to not respect .gitignores, it's easy: `alias rg="rg -u"`.

(To be clear, I agree that the default is a trade off. I'm not so gung-ho on this that I think it should never be anything else. But for my project and its goals, I think it's the best fit.)

Re: Ripgrep – A new command line search tool

#148

Post by raphlinus »

JoshTriplett wrote: post #138
ggreer wrote: post #63

I'm the author of ag. That was a really good comparison of the different code searching tools. The author did a great job of showing how each tool misbehaved or performed poorly in certain circumstances. He's also totally right about defaults mattering. It looks like ripgrep gets most of its speedup on ag by: 1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are…

> 1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are accustomed to full PCRE support. Switching would cause me to receive a lot of angry emails. Maybe I'll do it anyway. PCRE has some annoying limitations. (For example, it can only search up to 2GB at a time.) The standard trick here is to use the faster method for searches that it supports, and use the slowe…

Or use fancy-regex. Not ready for prime-time, but potentially the best of both worlds.

Re: Ripgrep – A new command line search tool

#149

Post by burntsushi »

nolemurs wrote: post #140
burntsushi wrote: post #84

Earlier quoted context omitted.

> and `grep` will be about as fast as anything I'd encourage you to check out the benchmarks in my blog, especially the subtitle benchmarks, because this isn't actually true in the case of ripgrep. :-)

I take it back! I'm definitely switching over to using ripgrep for the next bit. Off topic - is there a way to exclude directories? I tried doing things like `rg pattern -g '!my_dir/ ' but nothing I tried seemed to work.

Other than using your .gitignore or .ignore files to ignore it, that is indeed the intended way to do it.

Sadly, I have an embarrassing bug. Should be fixed soon: https://github.com/BurntSushi/ripgrep/issues/43

Post reply on HN