271–280 of 350 posts
I've done lots of work in this space, including computer vision and ML approaches, and Tabula[1] which was the gold standard for extraction.
PDF Plumber is better on just about every example I've tried.
Earlier quoted context omitted.
Actually, no thanks. "Sementic" structure is how we got responsive web soup of ugly websites with hamburger menus. We need the opposite, we need a format that stays the same size, same proportions and is vectorized so you can zoom to any size - however, the relationship of space between elements remains constant. PDF is an amazing format IMO. Think of it like Docker - the designer knows exactly how its going to appea…
So you're saying that if you have HTML as a tool then you're irresistibly drawn to making a complex page with menus and ugliness? I don't think that's a problem in HTML. I think the problem is in your head. > we need a format that stays the same size, same proportions Please explain how this helps people read a paper on screens of different sizes and proportions. Like me on my phone.
Earlier quoted context omitted.
Actually, no thanks. "Sementic" structure is how we got responsive web soup of ugly websites with hamburger menus. We need the opposite, we need a format that stays the same size, same proportions and is vectorized so you can zoom to any size - however, the relationship of space between elements remains constant. PDF is an amazing format IMO. Think of it like Docker - the designer knows exactly how its going to appea…
So you're saying that if you have HTML as a tool then you're irresistibly drawn to making a complex page with menus and ugliness? I don't think that's a problem in HTML. I think the problem is in your head. > we need a format that stays the same size, same proportions Please explain how this helps people read a paper on screens of different sizes and proportions. Like me on my phone.
Earlier quoted context omitted.
I’m the founder of a startup that is doing this, as well. We strive to be as simple and easy as possible to use. If you care to check us out: https://siftrics.com/
I like how your demo is clear and also in Linux! Surprising. The page is clear and easy to understand, looks good. Well done.
I like to say that anyone with a good old ThinkPad and an internet connection can mint fortunes and build empires :-)
Earlier quoted context omitted.
I work in the print industry and some clients have the naive idea they'll save money by formatting their own documents (naive because usually this just means a lot more work for us, which they end up paying for). We need some metadata to rearrange and sort PDF pages for mailing and delivery (such as name, address, and start/end page for that customer). Our general rule is you provide metadata in an external file to m…
I can cosign on this methodology. I used to work in an organization that used to build pdfs for accounting and licensing documentation. I used a proprietary tool (Planetpress :( ) to generate the documents using metadata from a separate input file (csv or xml) to determine what column maps to what field. Good thing about this was as you have already outlined: It allowed for some flexibility in what was acceptable inp…
The problem we have with a lot of client files is that they look fine but printers don't care about "look fine", they crash hard when they run out of virtual memory due to poor structure. And usually without a helpful error message, so that's more billable hours to diagnose. The most common culprit is workflows that develop single document PDFs then merge them resulting in thousands of similar and highly redundant subset fonts.
Earlier quoted context omitted.
That's why the only "reliable" way to extract text is to perform an OCR of the pdf rendering which is exactly what ABBYY is doing.
I’m the founder of a startup that is doing this, as well. We strive to be as simple and easy as possible to use. If you care to check us out: https://siftrics.com/
I am especially thinking about Japanese. Our company could probably find good uses of such service if it had Japanese support.
The fact that text might be oriented different wasn't covered in the article. IIRC Preview on Mac might search there (not near my mac ATM to check)
This is why iPhone didn't initially ship with double-tap to zoom for PDF paragraphs (like it had for blocks on web pages). I know because I was assigned the feature, and I went over to the PDF guy to ask how I would determine on an arbitrary PDF what was probably a "block" (paragraph), and I got a huge explanation on how hard it would be. I relayed this to my manager and the bug was punted. Edit: To add a little more…
> I know because I was assigned the feature, and I went over to the PDF guy to ask how I would determine on an arbitrary PDF what was probably a "block" (paragraph), and I got a huge explanation on how hard it would be. The funny thing is that creating a universal algorithm to convert PDFs and/or HTML to plaintext is probably comparable in difficulty to building level 5 self-driving cars, and would accrue at least as…
There's a lot more than zero dollars going into this... it's just that the end result is universally something that's "good enough for this one use-case for this one company" and that's as far as it gets.
Earlier quoted context omitted.
Not really. They import tabula_py, which is a Python wrapper around tabula-java (the library of which I'm a maintainer). Still, I would have loved at least a heads up from the team that sells Tabula Pro. I know they're not required to do so, but hey, they're kinda piggybacking on Tabula's "reputation".
You're being much more polite here than I would be. Even if it isn't illegal, what they've done is a giant dick move.
What do you recommend us to do, to not make you feel we made a dick move.
TIA
I'm an ML engineer, worked as a part time data engineer consultant for a medical lines/claims extraction company, for 3 years, which majorly involved in extracting the tabular data from the PDFs and Images. Developer rules or parsers as such is JUST no help. You end up creating a new rule every time you miss the data extraction. With that in consideration, and the existing resources are little help especially on skew…
I chuckled at your "the worst image" sample. Which still looked quite decent all things considered. You're "handwritten" example looks a bit "too decent" as well. I can see how that works. You first look for the edges of the table, and then you evaluate the symbol in each cell as something that matches unicode. So, how well does this cope with increasing degradation? i.e. pencil written notes that bleed outside cell…
"The Worst Image" is a close match to that, except it is a print.
Regarding increasing degradation - as stated above, the OCR engine is not proprietary - we confined ourselves to detect the structure at this moment, and started with the most common problems.