Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Table Detection and Extraction Using Deep Learning

nanonets.com

Post reply on HN
Print view View original

21–30 of 49 posts

Re: Table Detection and Extraction Using Deep Learning

#21

Post by nanoamp »

fny wrote: post #18
nanoamp wrote: post #16

I can see the use-case and potential for ML in exfiltrating tables, but I'd be worried about the potential for decision-making mistakes in environments the author identifies, such as finance. The example of TableNet using deep learning for table extraction on top of tesseract for OCR means two layers of ML, either of which could individually introduce pathologies without human oversight. It reminds me of the photocop…

There's a project from Microsoft Research that's really interesting which does just that: https://www.microsoft.com/en-us/research/publication/melford...

It looks like it's not quite the same thing, in that it identifies Excel values that should be formulae. It could be used in a pipeline with spreadsheets extracted by ML/OCR to reverse-engineer formulae though, which is an interesting prospect.

Re: Table Detection and Extraction Using Deep Learning

#22

Post by chriskanan »

With collaborators at Adobe Research, my lab published a paper recently showing how to do table reconstruction from infographics (e.g., bar charts) using deep learning [1].

While it isn't the sexiest project, I've had a number of companies reach out about the project. Human written rule-based approaches are pretty bad at the task, and even humans doing it manually aren't great (likely due to sloppiness).

[1] https://arxiv.org/abs/1908.01801

Re: Table Detection and Extraction Using Deep Learning

#23

Post by nightcracker »

crispyambulance wrote: post #20
willvarfar wrote: post #7

I've worked with several companies that try to parse things in PDF documents, extracting tables and paragraphs etc. This is actually challenging because a PDF is a large bag of words and fragments of words with x y positions. There is a particularly popular word processor that emits individual characters. Just determining that two fragments are part of the same word is challenging as is detecting bullet points, etc.…

It was really shocking when I learned that the way pdf works is as you describe, literally fragments of text with positions and essentially no semantics. I think a lot of folks find this out as I did, when they run into a project where they need to extract info from pdf documents. Without knowing anything about pdf, one can easily assume that it will be possible to do things like "can't we just extract some semantic…

It's so successful precisely because it doesn't have semantics. It is a print format with one goal: show the output as desired. Semantics only confuse and limit this goal.

Imagine how bogged down and limited vector graphics would be if every element had to have semantic meaning? "This line connects the of the to the 13th on the ".

Re: Table Detection and Extraction Using Deep Learning

#24

Post by Pandabob »

I applaud the Nanonet folks for starting a business around AI API's, and it seems clear there's lots of value to unlock with solutions like these.

But does anyone have insight how hard is it to be in a space where all the big cloud providers seem to be offering very similar products? Can you survive by focusing on a niche segment? Is the market growing so fast that there's room for multiple companies offering (roughly) the same thing?

Re: Table Detection and Extraction Using Deep Learning

#25

Post by ahpearce »

UglyToad wrote: post #10
willvarfar wrote: post #7

I've worked with several companies that try to parse things in PDF documents, extracting tables and paragraphs etc. This is actually challenging because a PDF is a large bag of words and fragments of words with x y positions. There is a particularly popular word processor that emits individual characters. Just determining that two fragments are part of the same word is challenging as is detecting bullet points, etc.…

I had to check we hadn't worked for the same company! Yeah, text extraction and layout analysis from PDFs is a super interesting challenge and still relatively underdeveloped. I'd put table detection at about the hardest challenge in that field. One of the contributors to the PDF library I'm developing has been implementing some interesting algorithms for layout analysis https://github.com/UglyToad/PdfPig/wiki/Docume…

Referring to the above poster's "non-locality", are we talking about denormalization of formatting? Is there a way to "normalize" PDF structure? Calculate margins or common formats beforehand to normalize?

Re: Table Detection and Extraction Using Deep Learning

#26

Post by UglyToad »

ahpearce wrote: post #25
UglyToad wrote: post #10

Earlier quoted context omitted.

I had to check we hadn't worked for the same company! Yeah, text extraction and layout analysis from PDFs is a super interesting challenge and still relatively underdeveloped. I'd put table detection at about the hardest challenge in that field. One of the contributors to the PDF library I'm developing has been implementing some interesting algorithms for layout analysis https://github.com/UglyToad/PdfPig/wiki/Docume…

Referring to the above poster's "non-locality", are we talking about denormalization of formatting? Is there a way to "normalize" PDF structure? Calculate margins or common formats beforehand to normalize?

I'll try my best to answer but I may be misunderstanding the question.

The current layout analysis algorithms don't do much normalization as far as I'm aware, the Recursive-XY Cut algorithm uses page level font-size information [0] to tune parameters but it doesn't infer a common structure or format either as an input or result.

The aim of most layout analysis algorithms is to produce classifications for regions, e.g. paragraphs, titles, lists which I suppose counts as denormalizing the document? Arriving at those classifications generally relies on first splitting the document into sections or regions and then classifying those regions. So far the implemented algorithms mainly focus on the first step, splitting a document into discrete regions. An example of the second step using ML approaches to classify those regions by the same contributor can be found here [1].

With the rule based approaches I've been experimenting with you can use certain information from the common producers to normalize certain features. For example line spacing and font size have a well defined relationship, as do whitespace size and font size (though this is a fuzzier relationship and goes out the window entirely for justified text).

An example where you rely on non-locality to parse a document, in this SEC filing there are both key values and a table: https://www.sec.gov/Archives/edgar/data/1428796/000110465920...

For the values following the subheading "Institutional Investment Manager Filing this Report:" the left hand column are keys for the right hand values.

At the bottom of the document there's a table containing the columns "Form 13F File Number" and "Name".

Now you could use a couple of rules to infer the difference between the key-values and the table:

1) The keys in a key value list end in ':'.

2) The keys in a key value list have a different font/color to the values.

Both of those rules hold true here but not in all or even most documents. For this reason you need to use the whole page to deduce the type of these sections, rather than immediately surrounding features/pixels as an ML algorithm might.

[0]: https://github.com/UglyToad/PdfPig/blob/master/src/UglyToad....

[1]: https://github.com/BobLd/PdfPigMLNetBlockClassifier

Re: Table Detection and Extraction Using Deep Learning

#27

Post by fallous »

ahpearce wrote: post #25
UglyToad wrote: post #10

Earlier quoted context omitted.

I had to check we hadn't worked for the same company! Yeah, text extraction and layout analysis from PDFs is a super interesting challenge and still relatively underdeveloped. I'd put table detection at about the hardest challenge in that field. One of the contributors to the PDF library I'm developing has been implementing some interesting algorithms for layout analysis https://github.com/UglyToad/PdfPig/wiki/Docume…

Referring to the above poster's "non-locality", are we talking about denormalization of formatting? Is there a way to "normalize" PDF structure? Calculate margins or common formats beforehand to normalize?

I believe the reference is to logical locality, specifically in the case of PDF that transforms and such are essentially atomic and there's no real boundary layer in which you may say "transform X and transform Y are equivalent within this local finite domain."

There really is no real differentiation between formatting and content in a PDF, so it's not possible to truly separate them.

Re: Table Detection and Extraction Using Deep Learning

#29

Post by bondolo »

This capability has a lot of value for accessibility. Recovering the table structure for logical presentation allows navigation by blind users as well as users who are not using a pointing device.

It is disappointing just how haphazardly most PDFs are structured. Too many of the PDF production tools remove all document structure metadata or fail to include it by default.

Re: Table Detection and Extraction Using Deep Learning

#30

Post by Ididntdothis »

Animats wrote: post #2

Table extraction has been a feature of better OCR programs for at least a decade. It's easier than the OCR part. Look up "OCR table" for examples, products, code, papers, etc.

Then go ahead, make a table extractor for PDF and get very rich. A lot of people have tried.
Post reply on HN