Why are all the amounts values negative?
bankstatementconverter.com
Why are all the amounts values negative?
1–10 of 24 posts
Re: Why are all the amounts values negative?
#2This is a great example of why a PDF’s text layer is not necessarily the same as its visible content.
I’d be cautious about making “non-black means invisible” the general rule. A possible middle ground between raw text extraction and full OCR would be to render the page, map each extracted glyph’s bounding box back to the rendered pixels, and discard glyphs that have almost no contrast with their local background.
That would preserve the exact embedded text for visible characters while using rasterization only to determine visibility.
Has anyone tried this kind of hybrid approach?
Re: Why are all the amounts values negative?
#3Does this end on a cliffhanger?
Did they try one method, and what was the feedback?
Re: Why are all the amounts values negative?
#4Does this end on a cliffhanger? Did they try one method, and what was the feedback?
I did try method number two, and it worked. I should update this post.
Re: Why are all the amounts values negative?
#5OCR the PDF.
It's the ground truth, and it's not like it's more complex than parsing a pdf, at this point, the technology for OCR might even be better than PDF parsing, which is full of accidental instead of natural complexity.
Re: Why are all the amounts values negative?
#6Why not skipping all the minus signs unless they do not immediately precede a number. I see all the minus signs are after the number, not before, so just skip if they are not before. Preferably in the same BT/ET block as the number.
Re: Why are all the amounts values negative?
#7Why not skipping all the minus signs unless they do not immediately precede a number. I see all the minus signs are after the number, not before, so just skip if they are not before. Preferably in the same BT/ET block as the number.
The trailing minus signs are true negatives.
Re: Why are all the amounts values negative?
#8I was expecting something about double-entry accounting and how it can (initially) be non-intuitive whether to mark a given entry as a debit (-100) or credit (+100), especially for liabilities (when you enter a credit card payment in your ledger, the signs aren't what you might intuitively expect when you haven't wrapped your head around the system yet).
Instead I get a nice blog post about a knotty little problem caused by someone's abuse of semantic markup for style purposes. Shows that HTML doesn't have a monopoly on that issue. PDFs are already bad enough as a format for extracting data from, we don't need dumb HTML-inspired tricks on top of that.
Nice job finding the issue and explaining it clearly, OP. Thanks for an interesting writeup.
Re: Why are all the amounts values negative?
#9Why not skipping all the minus signs unless they do not immediately precede a number. I see all the minus signs are after the number, not before, so just skip if they are not before. Preferably in the same BT/ET block as the number.
The trailing minus signs are true negatives.
Some trailing minus signs are true negatives, but it depends on the color which ones are and which ones aren't (and someone got paid for that. Amazing).
Re: Why are all the amounts values negative?
#10Why not skipping all the minus signs unless they do not immediately precede a number. I see all the minus signs are after the number, not before, so just skip if they are not before. Preferably in the same BT/ET block as the number.
That's the problem: what you see and what is actually in the PDF are not the same thing: the actual text in the PDF has a minus sign on both. The entry that you see as "$110.00-" is withdrawal, and its minus sign is very much real. However, the entry that you see as "$1,527.57" also has a minus sign. but you don't see it because it's been given the same font colour as the background.
So we have two identically formatted piece of data, where only the text colour offers a clue on what's real and what isn't. Absolutely insane.