51–60 of 88 posts
Impressive performance gains! 5x faster than MuPDF is significant, especially for applications processing large volumes of PDFs. Zig's memory safety without garbage collection overhead makes it ideal for this kind of performance-critical work. I'm curious about the trade-offs mentioned in the comments regarding Unicode handling. For document analysis pipelines (like extracting text from technical documentation or res…
I built a PDF text extraction library in Zig that's significantly faster than MuPDF for text extraction workloads. ~41K pages/sec peak throughput. Key choices: memory-mapped I/O, SIMD string search, parallel page extraction, streaming output. Handles CID fonts, incremental updates, all common compression filters. ~5,000 lines, no dependencies, compiles in Why it's fast: - Memory-mapped file I/O (no read syscalls) - Z…
What's fast about mmap?
You avoid an unnecessary copy. Normal read system call gets the data from disk hardware into the kernel page cache and then copies it into the buffer you provide in your process memory. With mmap, the page cache is mapped directly into your process memory, no copy.
All running processes share the mapped copy of the file.
There are a lot of downsides to mmap: you lose explicit error handling and fine-grained control of when exactly I/O happens. Consult the classic article on why sophisticated systems like DBMSs do not use mmap: https://db.cs.cmu.edu/mmap-cidr2022/
fpdf
jpdf
cpdf
cpppdf
bfpdf
ppdf
...
opdf
Earlier quoted context omitted.
What's fast about mmap?
Two big advantages: You avoid an unnecessary copy. Normal read system call gets the data from disk hardware into the kernel page cache and then copies it into the buffer you provide in your process memory. With mmap, the page cache is mapped directly into your process memory, no copy. All running processes share the mapped copy of the file. There are a lot of downsides to mmap: you lose explicit error handling and fi…
I now wonder which use cases would mmap suit better - if any...
> All running processes share the mapped copy of the file.
So something like building linkers that deal with read only shared libraries "plugins" etc ..?
https://github.com/Lulzx/zpdf/blob/main/python/tests/test_zp...
Earlier quoted context omitted.
fixed.
Yeah, sorry for confusion. When said Unicode, meant foreign text rather (just) the unescaped symbols, e.g. Greek. At one random Greek textbook[0], zpdf output is (extract | head -15): 01F9020101FC020401F9020301FB02070205020800030209020701FF01F90203020901F9012D020A0201020101FF01FB01FE0208 0200012E0219021802160218013202120222 0209021D0212021D012E013202200222000301FA021A0220021C022002160213012E0222000F000301F90206012C 0…
ΑΛΕΞΑΝΔΡΟΣ ΤΡΙΑΝΤΑΦΥΛΛΙΔΗΣ Καθηγητής Τμήματος Βιολογίας, ΑΠΘ
ΝΙΚΟΛΕΤΑ ΚΑΡΑΪΣΚΟΥ
Επίκουρη Καθηγήτρια Τμήματος Βιολογίας, ΑΠΘ
ΚΩΝΣΤΑΝΤΙΝΟΣ ΓΚΑΓΚΑΒΟΥΖΗΣ
Μεταδιδάκτορας Τμήματος Βιολογίας, ΑΠΘ
Γονιδιώματα
Δομή, Λειτουργία και ΕφαρμογέςImpressive performance gains! 5x faster than MuPDF is significant, especially for applications processing large volumes of PDFs. Zig's memory safety without garbage collection overhead makes it ideal for this kind of performance-critical work. I'm curious about the trade-offs mentioned in the comments regarding Unicode handling. For document analysis pipelines (like extracting text from technical documentation or res…
What memory safety?
- First commit 3hours ago. - commit message: LLM-generated. - README: LLM-generated. I'm not convinced that projects vibe coded over the evening deserve the HN front page… Edit: and of course the author's blog is also full of AI slop… 2026 hasn't even started I already hate it.
Wait, but why? If it's really better than what we had before, what does it matter how it was made? It's literally hacked together with the tools of the day (LLMs) isn't that the very hacker ethos? Patching stuff together that works in a new and useful way. 5x speed improvements on pdf text extraction might be great for some applications I'm not aware of, I wouldn't just dismiss it out of hand because the author used…
That's a very big if. The whole point is that what we had before was made slowly. This was made quickly. In itself it's not better but what it typically means is hours and hours of testing. Going through painful problems that highlight idiosyncrasies of the problem space. Things that are really weird and specific to whatever the tool is trying to address.
In such cases we can be expect that with very little time very few things were tested and tested properly (including a comment mentioned how tests were also generated). "We" the audience of potentially interested users have then to do that work (as plenty did commenting on that post).
IMHO what you bring forward is precisely that :
- can the new "solution" actually pass ALL the tests the previous one did? More?
This should be brought to the top and the actual compromises can then be understood, "we" can then decide if it's "better" for our context. In some cases faster with lossy output is actually better, in others absolutely not. The difference between the new and the old solutions isn't binary and have no visibility on that is what makes such a process nothing more than yet another showcase that LLMs can indeed produce "something" that is absolutely boring while consuming a TON of resources, including our own attention.
TL;DR: there should be test "harness" made by 3rd parties (or from well known software it is the closest too) that an LLM generated piece of code should pass before being actually compared.