Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is the best method for turning a scanned book as a PDF into text?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 122 posts

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#21

Post by pronoiac »

I made a high-quality scan of PAIP (Paradigms of Artificial Intelligence Programming), and worked on OCR'ing and incorporating that into an admittedly imperfect git repo of Markdown files. I used Scantailor to deskew and do other adjustments before applying Tesseract, via OCRmyPDF. I wrote notes for some of my process over at https://github.com/norvig/paip-lisp/releases/tag/v1.2 .

I'd also tried ocrit, which uses Apple's Vision framework for OCR, with some success - https://github.com/insidegui/ocrit

It's an ongoing, iterative process. I'll watch this thread with interest.

Some recent threads that might be helpful:

* https://news.ycombinator.com/item?id=42443022 - Show HN: Adventures in OCR

* https://news.ycombinator.com/item?id=43045801 - Benchmarking vision-language models on OCR in dynamic video environments - driscoll42 posted some stats from research

* https://news.ycombinator.com/item?id=43043671 - OCR4all

(Meaning, I have these browser tabs open, I haven't fully digested them yet)

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#22

Post by tyronehed »

For years I have been printing PDFs off on regular paper and then binding them into books. 1. Print it at work when no one is looking. 2. Get two rigid boards and squeeze the stack of paper together. I customarily use two wooden armrests that originally came from a garden-furniture lounger. 3. Squeeze the paper with just a 1/4-inch showing. 4. Use wood glue and with your finger working like a toothbrush, work the glue into the pages at the gluing end. 5. Get a 14-inch X 4-inch strip of canvas. I use cutoff painter's canvas. 6. Hang all this by the boards and put glue also on top of the canvas strip. 7. When it dries, remove the boards and glue down the sides. You have a strong, bound book out of those printed pages.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#25

Post by kingkongjaffa »

pronoiac wrote: post #21

I made a high-quality scan of PAIP (Paradigms of Artificial Intelligence Programming), and worked on OCR'ing and incorporating that into an admittedly imperfect git repo of Markdown files. I used Scantailor to deskew and do other adjustments before applying Tesseract, via OCRmyPDF. I wrote notes for some of my process over at https://github.com/norvig/paip-lisp/releases/tag/v1.2 . I'd also tried ocrit, which uses App…

Was technology the right approach here? Is it essentially done now? I couldn’t tell if it was completed entirely.

I can’t help but think a few amateur humans could have read the pdf with their eyes and written the markdown by hand if the OCR was a little sketchy.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#27

Post by aragonite »

I did this very recently for a 19th century book in German with occasionally some Greek. The method that produces the highest level of accuracy I've found is to use ImageMagick to extract each page as a image, then send each image file to Claude Sonnet (encoded as base64) with a simple user prompt like "Transcribe the complete text from this image verbatim with no additional commentary or explanations". The whole thing is completed in under an hour & the result is near perfect and certainly much better than from standard OCR softwares.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#28

Post by staplung »

maurycy wrote: post #17

I had very good experience with `gemini-2.0-flash-exp`: https://github.com/maurycy/gemini-json-ocr

It's hard to know what to make of this because while you've included the output JSON you haven't included the input PDF so I have no idea how to interpret what it's actually doing.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#30

Post by pronoiac »

kingkongjaffa wrote: post #25
pronoiac wrote: post #21

I made a high-quality scan of PAIP (Paradigms of Artificial Intelligence Programming), and worked on OCR'ing and incorporating that into an admittedly imperfect git repo of Markdown files. I used Scantailor to deskew and do other adjustments before applying Tesseract, via OCRmyPDF. I wrote notes for some of my process over at https://github.com/norvig/paip-lisp/releases/tag/v1.2 . I'd also tried ocrit, which uses App…

Was technology the right approach here? Is it essentially done now? I couldn’t tell if it was completed entirely. I can’t help but think a few amateur humans could have read the pdf with their eyes and written the markdown by hand if the OCR was a little sketchy.

It's still in progress! It's looong - about a thousand pages. There's an ebook, but the printed book got more editing.
Post reply on HN