Ask HN: How to use Machine Learning to extract facts from the text?

Post by dartwing » Mon, Apr 24, 2017, 2:22 PM UTC

lwhsiao wrote: post #15

DrNuke wrote: post #4

Looking for something similar for .pdf academic papers in my field but nothing really useful to automatise the extraction process exists, so the best path is still to extract data manually, homogenise data in a standard protocol, fed ML algos. Once a data protocol becomes a widespread standard and maybe a ISO or similar, there is a chance automated extraction will work at the finest level, as necessary for complex in…

On system for extracting information from PDFs is Fonduer[1], which is built on the Snorkel framework from Stanford. It may be worth checking out for your use case. Here's a blog post introducing it [2]. Disclosure: I worked on the project. [1] https://arxiv.org/abs/1703.05028 [2] https://hazyresearch.github.io/snorkel/blog/fonduer.html

Post by dartwing » Tue, Apr 25, 2017, 12:26 PM UTC

PaulHoule wrote: post #22

This system is a commercially oriented fact extraction system https://github.com/machinalis/iepy that can be trained to get the kind of performance you would see in a text extractor customized by the likes of BBN or Booz Alan Hamilton. You need 20,000 training samples to start getting good results.