Show HN: Parsing horse racing charts with Apache PDFBox

Post by tcho » Sat, Jul 08, 2017, 4:23 AM UTC

robinhowlett wrote: post #21

joosters wrote: post #9

Very interesting! I had never heard of Apache PDFBox before, I must give it a try. I have a similar program that parses horse racing PDFs from sites such as www.racehorserunner.com - which are of a much simpler format, but cause endless problems for me when the PDFs have layout problems. For example, issues like one column being too long and overlapping with another, e.g the last race on http://www.racehorserunner.co…

Yes I encountered similar issues but many of them were able to be solved. With PDFBox I was able to deal with the content at a very low level (on a per-character basis), so that when for instance building a String, I would insert a pipe character when the distance between adjacent characters was greater than the width of the space character and then detect that when translating to a certain field. See the convertToTe…

Post by vbuwivbiu » Sat, Jul 08, 2017, 5:09 AM UTC

sk5t wrote: post #5

vbuwivbiu wrote: post #4

Earlier quoted context omitted.

that's still presenting the pages in portrait orientation. I want the pages to be landscape and for the text to flow in at least 2 columns.

PDF is used primarily for pre-paginated media and does not reflow text; if the PDF author wanted the pages in landscape orientation, or using some other paper dimensions, he would have specified that. Same goes for margins and the like.

Post by amenghra » Sat, Jul 08, 2017, 7:55 AM UTC

userbinator wrote: post #18

maxxxxx wrote: post #10

I still don't understand how PDF could become one of the standards for publishing documents. Well structured content gets converted into PDF which loses most of that structure. And then a lot of work is done to guess that structure from PDF and convert it back to a better file format. It just shows that successful solutions don't have to be technically good.

The keyword is "publishing" --- as in, producing human-readable physical copies, not electronic ones. It just so happens that the format was relatively suitable for the latter too (because it actually looks like a printed document rendered on the screen --- unlike HTML or other formats around at the time), which is why that use-case became popular. PDF is basically a descendant of PostScript, which was designed to co…

Post by tunaoftheland » Sat, Jul 08, 2017, 3:14 PM UTC

richiverse wrote: post #17

As a python programmer, I found R's pdftools to be indispensable for messy text based PDFs. I couldn't find a python lib that worked as consistently across variously different formats.

Post by joosters » Sat, Jul 08, 2017, 6:43 PM UTC

jahewson wrote: post #23

joosters wrote: post #9

Very interesting! I had never heard of Apache PDFBox before, I must give it a try. I have a similar program that parses horse racing PDFs from sites such as www.racehorserunner.com - which are of a much simpler format, but cause endless problems for me when the PDFs have layout problems. For example, issues like one column being too long and overlapping with another, e.g the last race on http://www.racehorserunner.co…

PDFBox committer here, if you want even lower-level access to the page content stream, without anything 'clever' at all, check out the PDFGraphicsStreamEngine class, which is a superclass of the text extraction and rendering classes. Gives you access to the raw glyphs. You can override PageRenderer too, for visual debugging, e.g. render glyph bounding boxes. We have an interactive Swing PDFDebugger which does just th…