Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zpdf: PDF text extraction in Zig

github.com

Post reply on HN
Print view View original

61–70 of 88 posts

Re: Zpdf: PDF text extraction in Zig

#61

Post by littlestymaar »

chanbam wrote: post #34
mpeg wrote: post #19

Earlier quoted context omitted.

thanks, claude, I guess haha as others have commented, I think while this is a nice portfolio piece, I would worry about its longevity as a vibe coded project

If he made something legitimately useful, who cares how?

It seems that he didn't even test it before submitting though…

The author has created 30 new projects on github, in half a dozen different programming language, over the past month alone, and he also happen to have an LLM-generated blog. I think it's fair to say it's not “legitimately useful” except as a way for the author to fill his resume as he's looking for a job.

This kind of behavior is toxic.

Re: Zpdf: PDF text extraction in Zig

#63

Post by utopiah »

utopiah wrote: post #60
kingkongjaffa wrote: post #17

Earlier quoted context omitted.

Wait, but why? If it's really better than what we had before, what does it matter how it was made? It's literally hacked together with the tools of the day (LLMs) isn't that the very hacker ethos? Patching stuff together that works in a new and useful way. 5x speed improvements on pdf text extraction might be great for some applications I'm not aware of, I wouldn't just dismiss it out of hand because the author used…

> If it's really better than what we had before That's a very big if. The whole point is that what we had before was made slowly. This was made quickly. In itself it's not better but what it typically means is hours and hours of testing. Going through painful problems that highlight idiosyncrasies of the problem space. Things that are really weird and specific to whatever the tool is trying to address. In such cases…

related https://news.ycombinator.com/item?id=46437688

Re: Zpdf: PDF text extraction in Zig

#64

Post by forgotpwd16 »

lulzx wrote: post #58
forgotpwd16 wrote: post #35

Earlier quoted context omitted.

Yeah, sorry for confusion. When said Unicode, meant foreign text rather (just) the unescaped symbols, e.g. Greek. At one random Greek textbook[0], zpdf output is (extract | head -15): 01F9020101FC020401F9020301FB02070205020800030209020701FF01F90203020901F9012D020A0201020101FF01FB01FE0208 0200012E0219021802160218013202120222 0209021D0212021D012E013202200222000301FA021A0220021C022002160213012E0222000F000301F90206012C 0…

works now! ΑΛΕΞΑΝΔΡΟΣ ΤΡΙΑΝΤΑΦΥΛΛΙΔΗΣ Καθηγητής Τμήματος Βιολογίας, ΑΠΘ ΝΙΚΟΛΕΤΑ ΚΑΡΑΪΣΚΟΥ Επίκουρη Καθηγήτρια Τμήματος Βιολογίας, ΑΠΘ ΚΩΝΣΤΑΝΤΙΝΟΣ ΓΚΑΓΚΑΒΟΥΖΗΣ Μεταδιδάκτορας Τμήματος Βιολογίας, ΑΠΘ Γονιδιώματα Δομή, Λειτουργία και Εφαρμογές

Nice! Speed wasn't even compromised. Still 5x when benching. Also saw now there's page with tool compiled to wasm. Cool.

Re: Zpdf: PDF text extraction in Zig

#66

Post by littlestymaar »

DetroitThrow wrote: post #37
TZubiri wrote: post #32

Earlier quoted context omitted.

In my experience with parsing PDFs, speed has never been an issue, it has always been a matter of quality.

I tried a small PDF and got a memory error. It's definitely much faster than MuPDF on that file.

“The fastest PDF extractor is the one that crashes at the beginning of the file” or something.

Re: Zpdf: PDF text extraction in Zig

#67

Post by littlestymaar »

lulzx wrote: post #2

I built a PDF text extraction library in Zig that's significantly faster than MuPDF for text extraction workloads. ~41K pages/sec peak throughput. Key choices: memory-mapped I/O, SIMD string search, parallel page extraction, streaming output. Handles CID fonts, incremental updates, all common compression filters. ~5,000 lines, no dependencies, compiles in Why it's fast: - Memory-mapped file I/O (no read syscalls) - Z…

> I built

You didn't. Claude did. Like it did write this comment.

And you didn't even bother testing it before submitting, which is insulting to everyone.

Re: Zpdf: PDF text extraction in Zig

#68

Post by commandersaki »

kennethallen wrote: post #52
jeffbee wrote: post #12

Earlier quoted context omitted.

What's fast about mmap?

Two big advantages: You avoid an unnecessary copy. Normal read system call gets the data from disk hardware into the kernel page cache and then copies it into the buffer you provide in your process memory. With mmap, the page cache is mapped directly into your process memory, no copy. All running processes share the mapped copy of the file. There are a lot of downsides to mmap: you lose explicit error handling and fi…

you lose explicit error handling

I've never had to use mmap but this is always been the issue in my head. If you're treating I/O as memory pages, what happens when you read a page and it needs to "fault" by reading the backing storage but the storage fails to deliver? What can be said at that point, or does the program crash?

Re: Zpdf: PDF text extraction in Zig

#69

Post by lulzx »

forgotpwd16 wrote: post #64
lulzx wrote: post #58

Earlier quoted context omitted.

works now! ΑΛΕΞΑΝΔΡΟΣ ΤΡΙΑΝΤΑΦΥΛΛΙΔΗΣ Καθηγητής Τμήματος Βιολογίας, ΑΠΘ ΝΙΚΟΛΕΤΑ ΚΑΡΑΪΣΚΟΥ Επίκουρη Καθηγήτρια Τμήματος Βιολογίας, ΑΠΘ ΚΩΝΣΤΑΝΤΙΝΟΣ ΓΚΑΓΚΑΒΟΥΖΗΣ Μεταδιδάκτορας Τμήματος Βιολογίας, ΑΠΘ Γονιδιώματα Δομή, Λειτουργία και Εφαρμογές

Nice! Speed wasn't even compromised. Still 5x when benching. Also saw now there's page with tool compiled to wasm. Cool.

thanks! :)
Post reply on HN