Sounds like a poor man's version of recoll
https://www.lesbonscomptes.com/recoll/
A PDF in a Zip file, in an email attachment. recoll can index it and do OCR if you like
Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
71–80 of 144 posts
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#72Earlier quoted context omitted.
FZF + ripgrep is really killer for me. I don't even bother organizing my notes anymore, I just throw everything markdown files in a flat directory and then I have a script that uses FZF + ripgrep to search through it when I need it. I search by "last modified first" so unless I'm digging for something very old the results are instant. Code snippets, finances, TODO lists, cake recipes... It's all in there. I use the s…
Can you share your script.
I'd love some more info as well!
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#73How could I use Rga to search my browsing history?
Your browser history (if you use Chrome or Firefox at least) is stored in a SQLite database.
It looks like rga can handle SQLite out of the box, so just making sure your history .db file is visible to rga may be all you need.
You can also use my Datasette tool to get a web UI against your history, see https://docs.datasette.io/en/stable/getting_started.html#usi...
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#74thanks but it's way faster to have my stuff in G drive that way I can open a browser tab, wait 5 seconds for it to load, locate the new screen location of the search bar, click it, wait for javascript to finish loading so I can click the search bar, click it for real this time, mistype because there's some kind of contenteditable event jank, wait 5 seconds for my results to come up, fix the typo, and just have my res…
If you're using Duckduckgo, just search ''!drive search-term'' or ''search term !drive'' or ''search !drive term'' More !operators here - https://duckduckgo.com/bang
Firefox supports custom search engines, the most bang for the buck custom search engine must be https://duckduckgo.com/?q=%s with keyword being the letter d. Then you get all these 13000+ bangs without having to configure the custom search engines. E.g. write "d !drive term" in url bar. And "d !w hacker news" sends you directly to https://en.wikipedia.org/wiki/Hacker_News
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#75Developer of the tool here :) Glad to see it posted here, I still actively use it myself. Also check out the fzf integration in the README: https://github.com/phiresky/ripgrep-all/blob/master/doc/rga-...
Currently the main branch is undergoing a refactor to add support for having custom extractors (calling out to other tools), and more flexible chains of extractors.
Ripgrep itself has functionality integrated to call custom extractors with the `--pre` flag, but by adding it here we can retain the benefits of the rga wrapper (more accurate file type matchers, caching, recursion into archives, adapter chaining, no slow shell scripts in between, etc).
Sadly, during rewriting it to allow this, I kind of got hung up and couldn't manage to figure out how to cleanly design that in Rust. I'd be really glad if a Rust expert could help me out here:
In the currently stable version, the main interface of each "adapter" is `fn(Read, Write) -> ()`. To allow custom adapter chaining I have to change it to be `fn(Read) -> Read` where each chained adapter wraps the read stream and converts it while reading. But then I get issues with how to handle threading etc, as well as a random deadlock that I haven't figured out how to solve so far :/
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#76Can it (or any tool) perform proximity searches on scanned PDFs? E.g word1 within 20 words of word2, on scanned PDFs? (I think this is non trivial but very useful.)
Scanned PDFs only work well if they already have an OCR layer. There's some optional integration of rga with tesseract, but it's pretty slow and less good than external OCR tools.
ripgrep-all can do the same regexes as rg on any filetypes it supports. So you can could do something like --multiline and foo(\w+[\s\n]+){,20}bar
It won't work exactly like this, but something similar should do it:
--multiline enables multiline matching
* foo searches for foo
* \w+ searches for at least one word character
* [\W]+ searches for at least one space/nonword character like sentence marks
* {,20} searches for at most 20 iterations of the word-space combination bar searches for bar
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#77I always found useful something along the lines of pdftotext -layout file.pdf | grep -E ... for PDFs, good to see a Swiss Army knife utility for all sorts of file though!
rga uses pdftotext (from poppler) internally for pdfs, except wraps it in parallelization and a very fast cache layer, since you usually want to do multiple queries per file :)
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#78Can it (or any tool) perform proximity searches on scanned PDFs? E.g word1 within 20 words of word2, on scanned PDFs? (I think this is non trivial but very useful.)
If its a scanned PDF (essentially a collection of 1 image per page), there would need to be an OCR step to get some text out first. Tesseract would work for this.
Once that's done, you have all the options available to perform that search. But I don't know of a search tool that does the OCR for you. I did read a blog post of someone uploading PDFs to google drive (they OCR them on upload) as an easy way to do this.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#79Wonderful! pdfgrep is good but slow.
pdfgrep has a --cache option since a while ago :) Not sure why they don't enable it by default. Still, this is much faster.
Re: Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz
#80I love that we’re seeing fast & flexible solutions for personal search. I’ve recently been playing with Recoll for full-text-search on content. Since it indexes content up front, the search is pretty fast. It can also easily accommodate tag metadata on files. It would be interesting to consider how ripgrep based tools can fit into generically broad “search your database of content” workflows (as opposed to remember o…
FZF + ripgrep is really killer for me. I don't even bother organizing my notes anymore, I just throw everything markdown files in a flat directory and then I have a script that uses FZF + ripgrep to search through it when I need it. I search by "last modified first" so unless I'm digging for something very old the results are instant. Code snippets, finances, TODO lists, cake recipes... It's all in there. I use the s…
About a year ago, I discovered it was very helpful for me to have git branches ordered by "recently modified first":
From my `~/.gitconfig`:
[alias]
brt = "!git for-each-ref refs/heads --color=always --sort -committerdate --format='%(HEAD)%(color:reset);%(color:yellow)%(refname:short)%(color:reset);%(contents:subject);%(color:green)(%(committerdate:relative))%(color:blue);' | column -t -s ';'"