11–20 of 24 posts
Does this end on a cliffhanger? Did they try one method, and what was the feedback?
I did try method number two, and it worked. I should update this post.
Why not skipping all the minus signs unless they do not immediately precede a number. I see all the minus signs are after the number, not before, so just skip if they are not before. Preferably in the same BT/ET block as the number.
That's the problem: what you see and what is actually in the PDF are not the same thing : the actual text in the PDF has a minus sign on both. The entry that you see as "$110.00-" is withdrawal, and its minus sign is very much real. However, the entry that you see as "$1,527.57" also has a minus sign . but you don't see it because it's been given the same font colour as the background. So we have two identically form…
This is a great example of why a PDF’s text layer is not necessarily the same as its visible content. I’d be cautious about making “non-black means invisible” the general rule. A possible middle ground between raw text extraction and full OCR would be to render the page, map each extracted glyph’s bounding box back to the rendered pixels, and discard glyphs that have almost no contrast with their local background. Th…
I should have made it clear in the blog post, but the first thing I do when processing a PDF is I identify the document_type. Then I run special rules for that document_type.
So the "non-black means invisible" rule only applies when processing this document_type.
I was expecting something about double-entry accounting and how it can (initially) be non-intuitive whether to mark a given entry as a debit (-100) or credit (+100), especially for liabilities (when you enter a credit card payment in your ledger, the signs aren't what you might intuitively expect when you haven't wrapped your head around the system yet). Instead I get a nice blog post about a knotty little problem ca…
Earlier quoted context omitted.
I did try method number two, and it worked. I should update this post.
Is it not dangerous to ignore text based on its color? For this particular case it might work but looks like you have a generic service that is supposed to work with any bank statement.
OCR the PDF. It's the ground truth, and it's not like it's more complex than parsing a pdf, at this point, the technology for OCR might even be better than PDF parsing, which is full of accidental instead of natural complexity.
Another thing to consider is OCR works well for English but not so well for other languages.
OCR the PDF. It's the ground truth, and it's not like it's more complex than parsing a pdf, at this point, the technology for OCR might even be better than PDF parsing, which is full of accidental instead of natural complexity.
I don’t really understand how OCRing a PDF could be more accurate than reading the text nodes. Another thing to consider is OCR works well for English but not so well for other languages.
OCR of a digital image generated from those PDF draw commands only operates on the visible portions that are rendered.
Another case to be considered .. FOI requests for documents from "pain in the arse" government departments or law firms can result in "digital documents" that are deliberately generated to be difficult to process (go figure) .. eg: filled with circular and spaghetti draw commands that "print correctly" but are a nightmare horror show to parse.
( "technically compliant" is a legally compliant "fuck you" )
This is a great example of why a PDF’s text layer is not necessarily the same as its visible content. I’d be cautious about making “non-black means invisible” the general rule. A possible middle ground between raw text extraction and full OCR would be to render the page, map each extracted glyph’s bounding box back to the rendered pixels, and discard glyphs that have almost no contrast with their local background. Th…
If you use a real minus sign, and not an ASCII hyphen, it should be the same width as the corresponding plus sign in any reasonable font.
Use a plus sign and make that invisible.
Here is an idea: columns of figures should use a monospaced font.
You don't just want right or left edges to align, but all corresponding digit positions should align:
12,345
+ 4,001
= -------
16,346