Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
1–10 of 36 posts
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#2While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences.
DOI traversal would be required
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#3Glad if this tool/lib is useful to some. I'm happy to answer any and all questions!
A Kivy [1] based cross-platform GUI would be a nice addition at some point.
[1] http://kivy.org
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#4While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences. DOI traversal would be required
Good point, and I will definitely take a look at that!
Would you open an issue on Github, and perhaps reference a few papers?
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#5While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences. DOI traversal would be required
Good point, and I will definitely take a look at that! Would you open an issue on Github, and perhaps reference a few papers?
Sure, I'll do it this evening
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#6While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences. DOI traversal would be required
DOIs are great for humans, unfortunately they'll take you to the publishers webpage and I don't know of a standard way of getting an actual PDF from a DOI. Maybe with PLOS, I know they're good at serving up different versions (xml with a different accept header iirc).
Searching for something on the page that looks like a download PDF button and trying that might get you 80% of the way there, along with at least giving the user the remaining DOI urls to visit themselves.
This is actually really relevant to a lot of problems I see so if anyone has a general solution / a 90% solution then I'm all ears :)
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#7While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences. DOI traversal would be required
DOIs are great for humans, unfortunately they'll take you to the publishers webpage and I don't know of a standard way of getting an actual PDF from a DOI. Maybe with PLOS, I know they're good at serving up different versions (xml with a different accept header iirc). Searching for something on the page that looks like a download PDF button and trying that might get you 80% of the way there, along with at least givin…
The standard way for getting the actual PDF from a DOI, when it's a Crossref DOI (which it probably is) is to use the full-text link, available in the CrossRef API.
For DOI 10.1155/2010/963926
http://api.crossref.org/works/10.1155/2010/963926
From the returned JSON message -> link -> there's the PDF!
[
{
intended-application: "text-mining",
content-version: "vor",
content-type: "application/pdf",
URL: "http://downloads.hindawi.com/journals/jo/2010/963926.pdf"
},
{
intended-application: "text-mining",
content-version: "vor",
content-type: "application/xml",
URL: "http://downloads.hindawi.com/journals/jo/2010/963926.xml"
}
]
When a PDF has Crossref CrossMark, the DOI is embedded in the metadata (I can't say how but I can find out)
http://www.plosone.org/article/fetchObject.action?uri=info:d...
Drop us a line on labs@crossref.org
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#8While nice, it really only works with the reference has a direct link to the PDF while the majority of citations use the DOI in the sciences. DOI traversal would be required
DOIs are great for humans, unfortunately they'll take you to the publishers webpage and I don't know of a standard way of getting an actual PDF from a DOI. Maybe with PLOS, I know they're good at serving up different versions (xml with a different accept header iirc). Searching for something on the page that looks like a download PDF button and trying that might get you 80% of the way there, along with at least givin…
Maybe consider a Google Scholar integration? Its search results include links to full-text pdf at times. Even not, extracted links to publisher websites could be helpful for a batch review of referenced articles.
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#9Most PDF scientific articles sadly don't have good embedded metadata [0], so this & the DOI issue make this not very useful (at least for the journals I read).
I also would have implemented this with a simpler shell script calling exiftool[1] and pdftotext[2], but hey; fun to have a python-based implementation :)
[0] http://rossmounce.co.uk/2012/12/31/pdf-metadata-why-so-poor/ [1] http://www.sno.phy.queensu.ca/~phil/exiftool/ [2] http://poppler.freedesktop.org/
Re: Show HN: PDFx – Extract Metadata and URLs from PDFs, and Download Referenced PDFs
#10Earlier quoted context omitted.
DOIs are great for humans, unfortunately they'll take you to the publishers webpage and I don't know of a standard way of getting an actual PDF from a DOI. Maybe with PLOS, I know they're good at serving up different versions (xml with a different accept header iirc). Searching for something on the page that looks like a download PDF button and trying that might get you 80% of the way there, along with at least givin…
Maybe consider a Google Scholar integration? Its search results include links to full-text pdf at times. Even not, extracted links to publisher websites could be helpful for a batch review of referenced articles.
Not wishing to nit-pick, but a DOI is a link to the publisher website.