Lmgrep: Lucene-based grep-like utility

#1

Post by merename » Sat, Apr 24, 2021, 8:43 PM UTC

Lmgrep: Lucene-based grep-like utility
jocas.lt

Re: Lmgrep: Lucene-based grep-like utility

#2

Post by axiosgunnar » Sat, Apr 24, 2021, 11:46 PM UTC

Is there an actual link to the source code somewhere in that blog post?

Re: Lmgrep: Lucene-based grep-like utility

#3

Post by Kina » Sat, Apr 24, 2021, 11:53 PM UTC

axiosgunnar wrote: post #2

Is there an actual link to the source code somewhere in that blog post?

Yes, it is there as an in-text link:

"Then the most complicated part was to prepare executable binaries for different operating systems. Plenty of CPU, RAM, VirtualBox with Windows and macOS virtual machines, and here we go."

https://github.com/dainiusjocas/lucene-grep/releases/tag/v20...

Re: Lmgrep: Lucene-based grep-like utility

#4

Post by [deleted] » Sat, Apr 24, 2021, 11:53 PM UTC

axiosgunnar wrote: post #2

Is there an actual link to the source code somewhere in that blog post?

[deleted]

Re: Lmgrep: Lucene-based grep-like utility

#5

Post by 29athrowaway » Sun, Apr 25, 2021, 12:18 AM UTC

I think one of the best code search tools I've seen is the one here: https://source.chromium.org/chromium

I guess that's what Google uses internally? Is there an open source alternative?

Re: Lmgrep: Lucene-based grep-like utility

#6

Post by benatkin » Sun, Apr 25, 2021, 12:50 AM UTC

I'm surprised/impressed that it can build an index and search it in 29 milliseconds. Pretty clever solution!

Re: Lmgrep: Lucene-based grep-like utility

#7

Post by boyter » Sun, Apr 25, 2021, 2:02 AM UTC

Neat. This is similar to a tool I have been working on (but need to finish off) as I saw the same issue.

Except rather than build an index I brute forced the search each time. For most repositories it’s fast enough even with ranking.

https://github.com/boyter/cs For those interested it’s still very WIP with noticeable issues in TUI mode.

Re: Lmgrep: Lucene-based grep-like utility

#8

Post by darkteflon » Sun, Apr 25, 2021, 2:13 AM UTC

I’ve been looking for something exactly like this. I keep the complete text of articles I’ve enjoyed but, to-date, effective searching has meant spinning up an ES instance, which is painful. This is a specific use case that isn’t necessarily well-served by something like grep or ripgrep. I’ll definitely try this, thanks - looks very elegant.

Re: Lmgrep: Lucene-based grep-like utility

#9

Post by antman » Sun, Apr 25, 2021, 2:22 AM UTC

Very nice, examples are in the repository https://github.com/dainiusjocas/lucene-grep#text-analysis

Re: Lmgrep: Lucene-based grep-like utility

#10

Post by gravypod » Sun, Apr 25, 2021, 2:50 AM UTC

29athrowaway wrote: post #5

I think one of the best code search tools I've seen is the one here: https://source.chromium.org/chromium I guess that's what Google uses internally? Is there an open source alternative?

I think there's a company attempting to implement their own version of something similar. A very important part of search is also the understanding of language semantics. Something that is really cool for this is Kythe [0].

[0] - https://kythe.io/