Partially related - is this what someone could use to detect a sudoku grid? The spaces and the digits from a picture?
Table Detection and Extraction Using Deep Learning
11–20 of 49 posts
Re: Table Detection and Extraction Using Deep Learning
#12For what it's worth, at my previous place we built a YOLO based model for detecting paragraphs/tables/headlines/page layouts mixed with traditional rule based OCR/layout detection.
Re: Table Detection and Extraction Using Deep Learning
#13I've worked with several companies that try to parse things in PDF documents, extracting tables and paragraphs etc. This is actually challenging because a PDF is a large bag of words and fragments of words with x y positions. There is a particularly popular word processor that emits individual characters. Just determining that two fragments are part of the same word is challenging as is detecting bullet points, etc.…
I had to check we hadn't worked for the same company! Yeah, text extraction and layout analysis from PDFs is a super interesting challenge and still relatively underdeveloped. I'd put table detection at about the hardest challenge in that field. One of the contributors to the PDF library I'm developing has been implementing some interesting algorithms for layout analysis https://github.com/UglyToad/PdfPig/wiki/Docume…
Really really interesting, hadn't seen pdfpig before!
In the delicious pics of results I can see the bullets treated as one column and the paragraphs for each bullet point actually run together as single chunk of text?
What do you think about tackling bullets and indents?
Re: Table Detection and Extraction Using Deep Learning
#14Related: table extraction using mixed integer programming to encode constraints: https://news.ycombinator.com/item?id=21256005
Re: Table Detection and Extraction Using Deep Learning
#15Earlier quoted context omitted.
I had to check we hadn't worked for the same company! Yeah, text extraction and layout analysis from PDFs is a super interesting challenge and still relatively underdeveloped. I'd put table detection at about the hardest challenge in that field. One of the contributors to the PDF library I'm developing has been implementing some interesting algorithms for layout analysis https://github.com/UglyToad/PdfPig/wiki/Docume…
Really really interesting, hadn't seen pdfpig before! In the delicious pics of results I can see the bullets treated as one column and the paragraphs for each bullet point actually run together as single chunk of text? What do you think about tackling bullets and indents?
Thanks! I think there's definitely room for rules-based enhancements to the underlying algorithms.
My area of work on the project has been the core file-reading and file-creation stuff so I haven't had much of a chance to review the layout algorithm performance across documents.
Having been working on a purely rules-based approach in a private repository for a side project it seems like the algorithms the contributor has implemented get you a lot closer to the correct result than starting from rules alone but it definitely feels like adding some context-aware rules would get all the way there. I'm not sure whether they'd be in scope for the layout analysis project itself or someone could take the open-core and extend it, as I was attempting in my side project.
Re: Table Detection and Extraction Using Deep Learning
#16I can see the use-case and potential for ML in exfiltrating tables, but I'd be worried about the potential for decision-making mistakes in environments the author identifies, such as finance.
The example of TableNet using deep learning for table extraction on top of tesseract for OCR means two layers of ML, either of which could individually introduce pathologies without human oversight. It reminds me of the photocopier that changed numbers for you - https://www.theregister.co.uk/2013/08/06/xerox_copier_flaw_m...
If an ML engine was trained to be able to do things like look for totals and sub-totals in numerical tables and flag errors in summation, then that would clearly add more value in parsing for moderation (the use-case described at the end). But that doesn't seem to be something that's yet... on the table.
Re: Table Detection and Extraction Using Deep Learning
#17Table extraction has been a feature of better OCR programs for at least a decade. It's easier than the OCR part. Look up "OCR table" for examples, products, code, papers, etc.
You're wrong.Robust and easy to use table extraction might be solvable, but from a business perspective it isn't solved.
Did you try https://extracttable.com
The mentioned service is not perfect either. There are always limitations, minimizing is the key.
P.s: I work with the team at extracttable
Re: Table Detection and Extraction Using Deep Learning
#18I can see the use-case and potential for ML in exfiltrating tables, but I'd be worried about the potential for decision-making mistakes in environments the author identifies, such as finance. The example of TableNet using deep learning for table extraction on top of tesseract for OCR means two layers of ML, either of which could individually introduce pathologies without human oversight. It reminds me of the photocop…
There's a project from Microsoft Research that's really interesting which does just that:
https://www.microsoft.com/en-us/research/publication/melford...
Re: Table Detection and Extraction Using Deep Learning
#19I've worked with several companies that try to parse things in PDF documents, extracting tables and paragraphs etc. This is actually challenging because a PDF is a large bag of words and fragments of words with x y positions. There is a particularly popular word processor that emits individual characters. Just determining that two fragments are part of the same word is challenging as is detecting bullet points, etc.…
Yes exactly, table comprehension is a logic driven, non-local inference problem. Critically, its the non-locality that trips up common machine learning techniques. I wrote an approach using mixed integer programming once[1]
[1] https://edinburghhacklab.com/2013/09/probabalistic-scraping-...
Re: Table Detection and Extraction Using Deep Learning
#20I've worked with several companies that try to parse things in PDF documents, extracting tables and paragraphs etc. This is actually challenging because a PDF is a large bag of words and fragments of words with x y positions. There is a particularly popular word processor that emits individual characters. Just determining that two fragments are part of the same word is challenging as is detecting bullet points, etc.…
It was really shocking when I learned that the way pdf works is as you describe, literally fragments of text with positions and essentially no semantics.
I think a lot of folks find this out as I did, when they run into a project where they need to extract info from pdf documents. Without knowing anything about pdf, one can easily assume that it will be possible to do things like "can't we just extract some semantic structures like headings, tables, etc"... but nooo, it don't work that way!
Discovering the true nature of pdf is major WTF moment because we're so conditioned to expect documents to have a semantic structure. It's hard to understand how a standard can take the exact opposite approach and be so successful.