Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The surprisingly complex journey to text-selectable client-side generated PDFs

sdocs.dev

Post reply on HN
Print view View original

31–40 of 66 posts

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#31

Post by mvdwoord »

"everyone hates PDFs where you can't reliably select and copy text!"

Boy do I. One of my biggest annoyances is receiving an invoice in pdf format, where I can either not select the text at all, or where you cannot cleanly select text, i.e. when you try to select something it somehow half highlights the line above as well and I am not sure what is on my clipboard, and need to paste temporarily in a text editor, then select what I need ... etc

Super nice when the list IBAN numbers for payment in a tiny font size as well.

Maybe I should vibecode a little helper. tool to visually select a rectangle and perform OCR and detect IBAN numbers or show a popup with proper text to do my subselect.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#32

Post by adrian_b »

Worf wrote: post #10

PDFs should be only for printing or maybe for keeping scanned versions of things. For anything else they're just not the right tool for the job. Not for things meant to be accessed on a computer like books, scientific papers or, for some weird reason, catalogs and price lists from websites. We have responsive and open standards like HTML and EPUB (zipped XTML) and they work great. arXiv has HTML papers, and libgen an…

HTML and EPUB work not great, but very badly for scientific or technical papers or books.

No two readers render them alike, and they typically are much uglier and more difficult to use than books (sometimes even the same book) in PDF, DJVU or ODT formats.

I read a very large quantity of technical documentation and I always avoid EPUB and HTML like the plague. I use such formats only when there is no alternative.

On Linux, mupdf is a decent EPUB reader, which is very fast and it usually does a better job at formatting pages than most other EPUB readers that I have tried on Linux.

For fast navigation and searching, especially in technical documentation with hundreds or thousands of pages, it is very useful for the document to be well partitioned into pages and the page layouts to be well designed, like for a printed book, even if this may seem unnecessary for a document stored in a computer.

HTML and EPUB documents are seldom divided in uniform pages and the position of various elements, like tables or figures can vary between readers or even with the same reader in different circumstances, so when you search various things you are slowed down in recognizing them, because they may not be in the position where you have seen them previously. Moreover, in HTML and EPUB documents, depending on the reader, the size ratios between various elements may be inappropriate, making the pages ugly and/or hard to understand.

All the defects of HTML and EPUB documents are caused by the fact that the writer of the document normally does not take full responsibility for the appearance of the pages, delegating this to the browsers/readers, which seldom do a good job for scientific/technical documents full of formulae, tables and figures.

This may be fine for normal Web pages, but it is not acceptable for technical and scientific documents.

In theory, one could design carefully HTML pages and the associated CSS files, to be rendered deterministically, but I have encountered very rarely such documents.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#33

Post by yolo3000 »

Today I had to edit in MacOS a pdf which had some text fields. It had 3 places which resembled checkboxes, so in those I was able to place an `x` character. When I saved it and previewed it, the `x` was in 2 places, but not the third. I tried several times but couldn't make it work. I gave up, started a Claude session, and asked it to fill in the `x`. 4k tokens of Python later it managed to place the `x` correctly.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#34

Post by kuekacang »

Worf wrote: post #27
jkscm wrote: post #21

Earlier quoted context omitted.

slighlty disagree with this. A fixed page layout has it's own advantages. The reason we have more high quality pdf readers than epub readers is probrably connected to the format itself. PDF readers usally are more more feature complete when it comes to stuff like annotations too.

My issue with EPUB readers is about features a PDF reader wouldn't even have. Small and annoying things like how much freedom I get when changing fonts and whatnot. I haven't had a need to use annotations. I guess that could be solved by EPUB editors, but I haven't tested any, apart from any text editors after unzipping the EPUB.

Somewhat agree!. I think my main gripe is that there are no good-enough reader that: - without account integration - has good ux

Even some of the okay ones, most are, as tantacurl may say, janky.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#35

Post by 13324 »

mvdwoord wrote: post #31

"everyone hates PDFs where you can't reliably select and copy text!" Boy do I. One of my biggest annoyances is receiving an invoice in pdf format, where I can either not select the text at all, or where you cannot cleanly select text, i.e. when you try to select something it somehow half highlights the line above as well and I am not sure what is on my clipboard, and need to paste temporarily in a text editor, then s…

Personal lifehack: Use the address bar of your browser to view the clipboard content quickly and to omit any formatting from it.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#36

Post by Worf »

adrian_b wrote: post #32
Worf wrote: post #10

PDFs should be only for printing or maybe for keeping scanned versions of things. For anything else they're just not the right tool for the job. Not for things meant to be accessed on a computer like books, scientific papers or, for some weird reason, catalogs and price lists from websites. We have responsive and open standards like HTML and EPUB (zipped XTML) and they work great. arXiv has HTML papers, and libgen an…

HTML and EPUB work not great, but very badly for scientific or technical papers or books. No two readers render them alike, and they typically are much uglier and more difficult to use than books (sometimes even the same book) in PDF, DJVU or ODT formats. I read a very large quantity of technical documentation and I always avoid EPUB and HTML like the plague. I use such formats only when there is no alternative. On L…

I understand your point that you want a fixed presentation layout and pagination. I prefer to be able to responsively resize the document and to follow the ToC instead of pages. I've yet to see documentation that's thousands of pages long that doesn't include a very detailed ToC. For me remembering "section 8.3.16.2" is better and makes more sense than remembering "page 1292". I've had to read scanned math books years ago and I remember using some PDF reader to put bookmarks that correspond to the ToC, so I'd put "2.5.1.2 - Theorem about X" in the sidebar. That's how I was able to actually go back and forth easily. With just pages it would've taken me tens of seconds to locate a theorem (or lemma, proof, definition, whatever). And it was a dense book, so I had to constantly go back and reread stuff. But I agree that PDFs can, and often do, have ToCs, too.

> the size ratios between various elements may be inappropriate

I can't recall having this issue on websites or on EPUBs. What kinds of elements are we talking about? HTML and CSS are pretty good at keeping sizes from what I've seen. I agree that there are many EPUB readers, most of them very unpolished. And perhaps there aren't EPUB readers that are good at everything, yet.

For formulas, MathML and other tech has been satisfactory. I was able to find this basic math paper arXiv uses as a demo for their HTML papers:

https://ar5iv.labs.arxiv.org/html/1910.06709

It doesn't have figures, but the math is rendered perfectly. I can easily remove the "justify" style and increase and decrease the letters. If it was a long paper, it would've been nice to have a clickable ToC, but most EPUBs have one.

I think that right now most EPUB readers and some HTML renderings are bad, but I believe they'll get better.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#37

Post by Steve44 »

mvdwoord wrote: post #31

"everyone hates PDFs where you can't reliably select and copy text!" Boy do I. One of my biggest annoyances is receiving an invoice in pdf format, where I can either not select the text at all, or where you cannot cleanly select text, i.e. when you try to select something it somehow half highlights the line above as well and I am not sure what is on my clipboard, and need to paste temporarily in a text editor, then s…

We have a couple of large customers who will only send remittance advices as a PDF, the are several pages and a couple of hundred rows. Apparently their system can not send XLSX or any other format.

I've been a happy user of Tabula[1] for a few years and it works really well, for my needs anyway.

I just import, auto-detect tables, select "Stream", and then export to a CSV.

[1] https://tabula.technology/ [1] https://github.com/tabulapdf/tabula

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#38

Post by Worf »

jkscm wrote: post #30
Worf wrote: post #23

Earlier quoted context omitted.

The "figures that are deliberately placed close to the relevant section in the text" is something I've heard often, and I'd agree to an extent. But the figure is never 10 pages away (unless you have a tiny screen or something). It's easy to put an image inbetween 2 paragraphs. With PDF papers 1 figure is often referenced in several places throughout the paper so I just open 2 windows with the paper anyway. For justif…

Just look at examples like this: https://www.google.de/books/edition/Exploration_Map_by_Map/s... You will realize that saying "PDFs should be only for printing" is a vast oversimplifcation for the requirements people have for different kinds of documents.

I get "Keine Leseprobe verfügbar" which is "no sample available" according to DDG.

I'm downloading a 101 MB EPUB from Anna’s Archive. I'm not sure if it's an official EPUB or someone converted it from something else. There's no PDF available there. In a few hours I'll try to report how several readers handled it.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#39

Post by adrian_b »

Worf wrote: post #24
danhor wrote: post #15

Earlier quoted context omitted.

A PDF of a long document such as a standard or reference manual is almost always preferable to an HTML version. HTML versions have issues with formatting, searching (as browsers struggle with multi-thousand page documents and non-native search document search implementations almost always suck), indexing, correct behavior on windows size change (especially a side-by-side pdf view is almost unheard of for webpages), .…

> correct behavior on windows size change Except the PDF is not responsive at all and you can't increase or decrease the font size without increasing the whole width of page. > Some vendors have switched to online-only for some documents and it always annoys me. HTML shouldn't mean online-only. If the vendor isn't trying to make it hard to download, you should always be able to convert to PDF. But PDF to HTML is very…

In any technical/scientific document I do not want to increase or decrease the size of any element, e.g. of one of the fonts.

You only want to do an overall proportional zoom, when needed.

A well-designed document page has appropriate size ratios between various kinds of texts, formulae, tables and images, which should not be corrupted by changing the size of a single element.

The pages where the author has not formatted them adequately are ugly and hard to understand, which is what you typically see when this kind of content is written as HTML/EPUB documents, which are rendered non-deterministically.

Lazy writers may like HTML, but readers who must read and search through vast amounts of technical documentation do not like it.

There are many good PDF readers that are adequate for reading and searching even huge documents, but I have never seen any tool that works acceptably for EPUB/HTML big documents, which is not surprising because no tool can compensate the fact that the writer of the document did not design the layout of the pages carefully.

Re: The surprisingly complex journey to text-selectable client-side generated PDFs

#40

Post by jkscm »

Worf wrote: post #38
jkscm wrote: post #30

Earlier quoted context omitted.

Just look at examples like this: https://www.google.de/books/edition/Exploration_Map_by_Map/s... You will realize that saying "PDFs should be only for printing" is a vast oversimplifcation for the requirements people have for different kinds of documents.

I get "Keine Leseprobe verfügbar" which is "no sample available" according to DDG. I'm downloading a 101 MB EPUB from Anna’s Archive. I'm not sure if it's an official EPUB or someone converted it from something else. There's no PDF available there. In a few hours I'll try to report how several readers handled it.

It's also on amazon https://www.amazon.com/Exploration-Map-Migrations-Encounters...

There is a kindle version (which would be closer to epub than pdf) but it just is even worse than a pdf and the authors also include the reason for that:

> Due to the complex integration of images and text, this DK eBook has been formatted to retain the design of the print edition. As a result, all elements are fixed in place, but can easily be enlarged by using the pinch-to-zoom function.

EPUB is simply not ready for complex layouts.

Post reply on HN