Aww hell yeah we should definitely use this in place of ripgrep for the new ArchiveBox.io full-text search backend.
Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
81–90 of 144 posts
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#82Developer of the tool here :) Glad to see it posted here, I still actively use it myself. Also check out the fzf integration in the README: https://github.com/phiresky/ripgrep-all/blob/master/doc/rga-... Currently the main branch is undergoing a refactor to add support for having custom extractors (calling out to other tools), and more flexible chains of extractors. Ripgrep itself has functionality integrated to call…
> In the currently stable version, the main interface of each "adapter" is `fn(Read, Write) -> ()`. To allow custom adapter chaining I have to change it to be `fn(Read) -> Read` where each chained adapter wraps the read stream and converts it while reading. But then I get issues with how to handle threading etc, as well as a random deadlock that I haven't figured out how to solve so far :/
I don't quite grok the problem here. If you file an issue against ripgrep proper with code links and some more details, I can try to assist.
Taken literally, ripgrep uses that exact same approach. There are potentially multiple adapters being used. Each adapter is just defined to wrap a `std::io::Read` implementation, and the adapter in turn implements `std::io::Read` so that it can be composed with others. The part that I'm missing is why this has anything to do with threading or deadlocks. I/O adapters shouldn't be having anything to do with synchronization. So I'm probably misunderstanding your problem.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#83Developer of the tool here :) Glad to see it posted here, I still actively use it myself. Also check out the fzf integration in the README: https://github.com/phiresky/ripgrep-all/blob/master/doc/rga-... Currently the main branch is undergoing a refactor to add support for having custom extractors (calling out to other tools), and more flexible chains of extractors. Ripgrep itself has functionality integrated to call…
One possibility is the almost dirt-simple solution wherein you just have a "make"/"Makefile" (or your favorite other build system) maintain a shadow tree of parallel pre-translated files. You get parallelism via `make -j$(nproc)` or its equivalent.
Every name in the shadow is built from the name in the origin but maybe with ".txt" added (or .txt.gz if you want to keep the compressed with whatever is the fastest decompressor builtin to ripgrep as a library not called as a program). Untranslated names can be just symbolic/hard links back to the origin. Build rules become as flexible as your build system.
This also scales to deployments that have more disk space than memory. Admittedly, in that case, the whole procedure probably becomes disk-IO bound, but maybe not. Maybe some translations cannot even keep up with disk IO - NVMe storage is pretty fast, for example. Or available memory may vary dynamically a lot, sometimes allowing the shadow to be fully in the buffer cache, other times not. It strikes me as less presumptuous to assume you can find disk space vs. having that much memory available. (EDIT2: though I may be confused about how `rga` operates - your doc says "memory cache", though.)
On the pro-side, but for updating the shadows based on origins, the user could even just `rg` from within the shadow and translate filenames "in their head", although stripping an always present string is obviously trivial. Indeed, you won't need `rg --pre` at all and the grep itself could become pluggable. I doubt any of your other `fzf`/etc. integrations would be made more complicated by this design, either.
This all strikes me as simple/nice enough that someone has probably already done it...EDIT1: Oh, I see from thumbs ups and other comments over at [1] and [2] that @phiresky is probably already aware of this design idea, but maybe some HN person knows of an existing solution along these lines.
[1] https://github.com/BurntSushi/ripgrep/issues/978 [2] https://github.com/BurntSushi/ripgrep/pull/981
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#84Earlier quoted context omitted.
FZF + ripgrep is really killer for me. I don't even bother organizing my notes anymore, I just throw everything markdown files in a flat directory and then I have a script that uses FZF + ripgrep to search through it when I need it. I search by "last modified first" so unless I'm digging for something very old the results are instant. Code snippets, finances, TODO lists, cake recipes... It's all in there. I use the s…
Can you share your script.
This is the main one (that actually only uses FZF, not ripgrep): https://gist.github.com/simias/b1d8356469d2a9386deeb7c45984b...
You'll need to set NOTES_DIR in your environment to wherever you want your notes to be stored. Then you can write `note something` to create or open $NOTES_DIR/something.md with your $EDITOR.
If you type "note" without parameter you'll start a search on all the note names, ordered by last use. If you type "note -f" it starts a full text search.
For best results you should have the fzf.vim's preview.sh somewhere in your fs, otherwise it'll use "cat" but it won't be as good looking (see FZF_PREVIEW in the script).
Hopefully despite being shell it should be readable enough to tweak to your liking.
Note that it was written and used exclusively on Linux, but I did try to avoid GNU-isms so hopefully it should work on BSDs and maybe even on MacOS with a bit of luck.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#85Earlier quoted context omitted.
FZF + ripgrep is really killer for me. I don't even bother organizing my notes anymore, I just throw everything markdown files in a flat directory and then I have a script that uses FZF + ripgrep to search through it when I need it. I search by "last modified first" so unless I'm digging for something very old the results are instant. Code snippets, finances, TODO lists, cake recipes... It's all in there. I use the s…
About a year ago, I discovered it was very helpful for me to have git branches ordered by "recently modified first": From my `~/.gitconfig`: [alias] brt = "!git for-each-ref refs/heads --color=always --sort -committerdate --format='%(HEAD)%(color:reset);%(color:yellow)%(refname:short)%(color:reset);%(contents:subject);%(color:green)(%(committerdate:relative))%(color:blue); ' | column -t -s ';'" I always spent a lot o…
Oh that's a great idea, I'd definitely stealing that, thanks!
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#86Big fan of rga! I use it almost every day for the academic part of my life, when I want to know the location of some specific keywords in my lecture slides, books or papers I've been reading. Even for single ebooks, it is often more useful than the search in Acrobat Reader.
> search in Acrobat Reader The search in PDF viewers is an anti-feature in terms of UI and performance. Their advantage is that they allow to scroll to and highlight the found phrase back in the document.
The search in Tracker Software's PDF X-Change Viewer/Editor is really great. Effective and easy to use
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#87One a related note there is one program that I absolutely miss on Linux called everything (on windows). The closest I can find is mlocate but it does not have a GUI but more importantly it does not index my Windows or NTFS drives. Would appreciate any suggestions if someone knows something like 'everything' for Ubuntu.
'Everything' is a LIFE SAVER.
Hmm.. I seem to remember creating an excel file for this client a while back.. open Everything -> filter client.xlsx .. boom. Or maybe I didn't name it properly, at all? Well still just a simple '*.xlsx' and sort by date, I can generally find anything this way. As long as you let Everything open on windows startup, it will be instant after use.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#88Idea behind Rga is cool.
Anyway, I tried it on Mac and installed via Homebrew. The formula already says it depends on ripgrep (that's fine since I have ripgrep already installed and use it regularly). I still was surprised when I executed Rga for the first time and got an error message that 'pdftotext' was not found. Since pdftotext has been officially discontinued, I am not sure if I want to install an old version just to make Rga work on my machine. Don't think it's an good idea to rely on a project which is not maintained actively.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#89Big fan of ripgrep. Use it on Windows to search through 100s of GBs of data really quickly.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#90Earlier quoted context omitted.
Works on non-Windows. ripgrep is notoriously fast. Command line interface. Not comically priced at 59.95 USD.
Why is there an expectation that every application should be free or cheap? IMHO $60 is very reasonable for a program that can save a lot of time for the user. And developers also have to eat, and might want to some day retire.
I'm not the boss of you. If you want to spend 60 USD on a program that is mostly built into Finder and Nautilus, fill your boots.