One nit in the repo README - you might want to change the cost reporting to be as $15 / 1000 pages instead of documents.
Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
81–90 of 97 posts
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#82Earlier quoted context omitted.
You've got a point, but try testing it on a tricky example like the Apollo 17 document - you know, with those sideways tables and old-school writing. You'll see all three non-AI services totally bomb. Now, if you tweak it to batch = 1 instead of 10, you'll notice there's hardly any made-up stuff. When you dial down the temperature close to zero, it's super unlikely to see hallucinations with limited context. At worst…
I will say I have had a look at your code here. I really do value your innovation here in gaining better accuracy, but I don't think it's is much more accurate for obscure PDF cases - Maybe it halves those obscure errors. I found it still hallucinated or failed to parse some text (e.g. that unusual languages, screenshots with tiny blurred JPEG text, images/shapes remain hallucination issues with your solution). BTW I…
Re: obscure PDFs, I’d love to see a PDF dataset with a whole bunch of these from different domains.
I think in general it’s very hard to say if any approach is “good enough” until you see some serious degree of variability in the input domain.
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#83Earlier quoted context omitted.
There are private APIs that have that data (now and history) Do you think the official data published is 100% correct if they were trying to hide something?
I am honestly not certain why they obstruct easy access to the number history. It's obviously accessible, but only through manually parsing the PDF. Their prior embedded search function, approximately two years ago, would return all permutations of the queried number from day 1 to present. They modified it to exclude results more than two years old. The PDF contains the entire data set, but isn't searchable. Why? Dun…
I've worked in the field and it could just be that the developers in charge of the new site didn't know/care how to get the data from the old system.
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#84I had been using GPT4o for extracting insights from Scanned docs, it was doing fine. But very recently (since they launched new model - o1), it's not working. GPT4o is refusing to extract text from images and says it can't do it, though it was doing same thing with same prompts till last week. I am not sure if this is intentional downgrade and it can be clubbed with new model launch, but it's really frustrating for m…
I have observed a lot of similar contradictions, where the large lout insists it can't do something that it did many times 'last week'.
Super frustrating when really trying to accomplish something!
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#85Earlier quoted context omitted.
I am honestly not certain why they obstruct easy access to the number history. It's obviously accessible, but only through manually parsing the PDF. Their prior embedded search function, approximately two years ago, would return all permutations of the queried number from day 1 to present. They modified it to exclude results more than two years old. The PDF contains the entire data set, but isn't searchable. Why? Dun…
I've worked in the field and it could just be that the developers in charge of the new site didn't know/care how to get the data from the old system.
Old Hanlon's razor. Maybe, but I'd rather assume malice when it comes to Florida Lottery
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#86I used GPT4o to convert heavily convoluted PDFs into csv files. The files were Florida Lottery Pick(n) histories, which they deliberately convolute to prevent automatic searching; ctrl-f does nothing and a fsck-ton of special characters embellish the whole file. I had previously done so manually, with regex, and was surprised with the quality of the end results of GPT, despite many preceding failed iterations. The wo…
I had the same problem with a PDF schematic for a BTT Octopus 3d printer board (which is published on their Github repo).
Unsearchable, weird characters behind the curtain, and etc.
But I don't blame deliberate obfuscation (or any other deliberate attempt to hide information) at all.
Instead, I simply blame incompetence.
(There's a ton of shitty PDFs in the world; this is just an example that I've encountered recently.)
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#87I have not found any mention of accuracy. Since it's using LLM, how accurate the conversion is? As in does that NASA document match 100% with the pdf or did it introduce any made up things (hallucinations)? That converted NASA doc should be included in repo and linked in readme if you haven't already.
People are really freaked out about hallucinations, but you can totally tackle that with solid prompts. The one in the repo right now is doing a pretty good job. Keep in mind though, this project is all about maxing out context for LLMs in products that need PDF input. We're not talking about some hardcore archiving system for the Library of Congress here. The goal is to boost consistency whenever you're feeding PDF…
> People are really freaked out about hallucinations, but you can totally tackle that with solid prompts.
> The goal is to boost consistency whenever you're feeding PDF context into an LLM-powered tool.
These two assertions are contradictory.
There are no "solid prompts" which obviate anthropomorphic "LLM hallucinations." Also, there is no deterministic consistency when "feeding PDF context" into an intrinsically non-deterministic algorithm, as any "LLM-powered tool" is by definition.
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#88There is also LLMWhisperer, a document pre-processor specifically made for LLM consumption.
As other mentioned, accuracy is the one part of solution criteria, other include, how does the preprocessing engine scale/performs at large scale, and how does it handle very complex documents like, bank loan forms with checkboxes, IRS tax forms with multi-layered nested tables etc.
https://unstract.com/llmwhisperer/
LLMWhisperer is a part of Unstract - An open-source tool for unstructured document ETL.
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#89Earlier quoted context omitted.
> I'm currently solving this problem for work and thinking of a spin out, what's a ballpark figure you'd be willing to pay per 1000 pages for 99.999% character level accuracy? I guess anything up to 5 ¢ per page would be acceptable. But I'm afraid my company wouldn't be a customer. We are in Germany and we deal with particularly protected private data, there is no chance that we would exfiltrate this data to a cloud…
What's the total spend per quarter? For a margin that fat I'd be willing to jump through a lot of hoops if you're doing enough pages. The models (currently) fit in 24gb vram sequentially with small enough batch sizes, so a local server with consumer grade gpus wouldn't be impossible.
I'll check and get back to you. How can I reach you?
Re: Show HN: PDF to MD by LLMs – Extract Text/Tables/Image Descriptives by GPT4o
#90You could transform arXiv to a markdown website