Llama-Scan: Convert PDFs to Text W Local LLMs
1–10 of 86 posts
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#2Almost perfect, the PDF I tested it missed only a few symbols.
But that is something I will use for sure. Thank you.
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#3Looking at the code, this converts PDF pages to images, then transcribes each image. I might have expected a pdftotext post-processor. The complexity of PDF I guess ...
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#4Looking at the code, this converts PDF pages to images, then transcribes each image. I might have expected a pdftotext post-processor. The complexity of PDF I guess ...
There is a very popular Python module called ocrmypdf. I used it to help my HOA and OCR’ing of old PDFs.
https://github.com/ocrmypdf/OCRmyPDF
No LLMs required.
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#5Ironically, Ollama likely is using Tesseract under the hood. Python library ocrmypdf uses Tesseract too.
https://github.com/ocrmypdf/OCRmyPDF
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#6Which llama model would have the best results for transcribing an image, I wonder. Say, for a screen grab of a newspaper page.
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#7Looking at the code, this converts PDF pages to images, then transcribes each image. I might have expected a pdftotext post-processor. The complexity of PDF I guess ...
Shell: GNU parallel, pdftotext
Python: PyPdf2, PdfMiner.six, Grobid, PyMuPdf; pytesseract (C++)
paperetl is built on grobid: https://github.com/neuml/paperetl
annotateai: https://github.com/neuml/annotateai :
> annotateai automatically annotates papers using Large Language Models (LLMs). While LLMs can summarize papers, search papers and build generative text about papers, this project focuses on providing human readers with context as they read.
pdf.js-hypothes.is: https://github.com/hypothesis/pdf.js-hypothes.is:
> This is a copy of Mozilla's PDF.js viewer with Hypothesis annotation tools added
Hypothesis is built on the W3C Web Annotations spec.
dokieli implements W3C Web Annotations and many other Linked Data Specs: https://github.com/dokieli/dokieli :
> Implements versioning and has the notion of immutable resources.
> Embedding data blocks, e.g., Turtle, N-Triples, JSON-LD, TriG (Nanopublications).
A dokieli document interface to LLMs would be basically the anti-PDF.
Rust crates: rayon handles parallel processing, pdf-rs, tesseract (C++)
pdf-rs examples/src/bin/extract_page.rs: https://github.com/pdf-rs/pdf/blob/master/examples/src/bin/e...
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#8Please add a license file. Thanks!
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#9Looking at the code, this converts PDF pages to images, then transcribes each image. I might have expected a pdftotext post-processor. The complexity of PDF I guess ...
I imagine part of the issue is how many PDFs are just a series of images anyway.
Re: Llama-Scan: Convert PDFs to Text W Local LLMs
#10Nice! I wonder what is the hardware required to run qwen2.5vl locally. A 6gb 2cpu VPS can do?