Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is the best method for turning a scanned book as a PDF into text?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 122 posts

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#61

Post by Electricniko »

For classic books like those you mentioned, Project Gutenberg has text versions along with pdfs/epubs/etc.

For instance, Discourses on Livy:

https://www.gutenberg.org/cache/epub/10827/pg10827-images.ht...

https://www.gutenberg.org/ebooks/10827

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#62

Post by jmrm »

Some time ago I was toying around with a library called [MuPDF](https://www.mupdf.com/) for something related, and with that library and a small Python script you can programmatically OCR any book you want.

That library is free for personal or open source projects, but paid for commercial ones

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#63

Post by tyrust »

Electricniko wrote: post #61

For classic books like those you mentioned, Project Gutenberg has text versions along with pdfs/epubs/etc. For instance, Discourses on Livy: https://www.gutenberg.org/cache/epub/10827/pg10827-images.ht... https://www.gutenberg.org/ebooks/10827

Even better is when Standard Ebooks publishes a version: https://standardebooks.org/ebooks/niccolo-machiavelli/discou...

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#64

Post by dotancohen »

jazzyjackson wrote: post #53
ant6n wrote: post #52

Earlier quoted context omitted.

I recently did some OCRing with OpenAI. I found o3-mini-hi to be imagining and changing text, whereas the older (?) o4 was more accurate. It’s a bit worrying that some of the models screw around with the text.

There’s GPT4, then GPT4o (o for Omni, as in multi modal) and then GPT o1 (chain of thought / internal reasoning) then o3 (because o2 is a stadium in London that I guess is very litigious about its trademark?), o3-mini is the latest but yes optimized to be faster and cheaper

What is the o3 model good for? Is it just an evolution of o1 (chain of thought / internal reasoning)?

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#65

Post by jonnycoder »

I recently used AWS Textract and had good results. There are accuracy benchmarks out there, I wish I saved the links, but I recall Gemini 2.0 and Textract towards the top in terms of accuracy. I also read that an LLM could extrapolate/conjure up cropped text therefore my idea would be to combine traditional OcR with LLM to determine conflicts.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#66

Post by jazzyjackson »

polshaw wrote: post #58
jazzyjackson wrote: post #53

Earlier quoted context omitted.

There’s GPT4, then GPT4o (o for Omni, as in multi modal) and then GPT o1 (chain of thought / internal reasoning) then o3 (because o2 is a stadium in London that I guess is very litigious about its trademark?), o3-mini is the latest but yes optimized to be faster and cheaper

o2 is the UK's largest mobile network operator. They bought naming rights to what was known as the millennium dome (not even a stadium).

Ahh makes sense :)

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#67

Post by eigenvalue »

I made a site recently that works pretty well for this for a lot of sample scanned PDFs I tried, you might get good results:

https://fixmydocuments.com/

I also made a simple iOS app that basically just uses the built in OCR functionality on iPhones and automatically applies it to all the pages of a PDF. It won’t preserve formatting, but it’s quite accurate in terms of OCR:

https://apps.apple.com/us/app/super-pdf-ocr/id6479674248

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#68

Post by carlosjobim »

__rito__ wrote: post #31

I have tried a bunch of things. This is what worked best for me: Surya [0]. It can run fully local on your laptop. I also tried EasyOCR [1], which is also quite good. I haven't tried this myself, but I will look at Paddle [2] if the previous two don't float your boat. All of these are OSS, and you don't need to pay a dime to anyone. [0]: https://github.com/VikParuchuri/surya [1]: https://github.com/JaidedAI/EasyOCR […

I would like to pay a dime and more for any of these solutions discussed in the thread as a normal MacOS program with a graphical user interface.

Re: Ask HN: What is the best method for turning a scanned book as a PDF into text?

#69

Post by woile »

aragonite wrote: post #27

I did this very recently for a 19th century book in German with occasionally some Greek. The method that produces the highest level of accuracy I've found is to use ImageMagick to extract each page as a image, then send each image file to Claude Sonnet (encoded as base64) with a simple user prompt like "Transcribe the complete text from this image verbatim with no additional commentary or explanations". The whole thi…

What about preserving the style like titles and subtitles?
Post reply on HN