71–80 of 101 posts
Earlier quoted context omitted.
I'm using Ubuntu 22.04. I encountered several errors with Poetry and attempted to fix them but eventually gave up.
(author) Please feel free to open an issue if you try again. Poetry can be painful, I might just switch to a requirements.txt file in the future. (you can skip poetry if you want by just pulling everything in pyproject.toml into a requirements.txt file also)
Impressive. It would be nice to have access to a spellchecker with support for more languages though. But the results are pretty good despite that.
Spellchecker is included. Just change the spell_Lang from eng to your lang
I'm curious if anyone has had any success building this package. I've spent a lot of time trying to build it myself, but unfortunately haven't been able to get it to work. Has anyone else had better luck?"
Random old issue for example: https://thetech.com/issues/33/34
Earlier quoted context omitted.
I'm using Ubuntu 22.04. I encountered several errors with Poetry and attempted to fix them but eventually gave up.
(author) Please feel free to open an issue if you try again. Poetry can be painful, I might just switch to a requirements.txt file in the future. (you can skip poetry if you want by just pulling everything in pyproject.toml into a requirements.txt file also)
Let's not underestimate the impact of such tool: we are talking about freeing up tons of knowledge from a "good for consumption/bad for distribution" format. I'm very excited about it. Let's build a pipeline: all the pdfs -> markdown them all -> archive.org them all
I don't think that is the right approach for archiving. The preferred pipeline would be
all the pdfs -> archive them all -> markdown them
This way you can always re-run the conversion as bugs are fixed and improvements are made. Generally archivist prefer to save as close to the source material as possible, because every transformation from there can only lose data.
Great work! I am a bit confused with the comparison with nougat throughout the repo. Nougat was specifically trained for academic documents, and I don't think anyone ever claimed Nougat was the best OCR model out there. That's kinda clear in your benchmark too where you mention that nougat has higher accuracy on arxiv documents. You also mention that marker will convert fewer equations when compared to nougat, and ye…
Great work! I am a bit confused with the comparison with nougat throughout the repo. Nougat was specifically trained for academic documents, and I don't think anyone ever claimed Nougat was the best OCR model out there. That's kinda clear in your benchmark too where you mention that nougat has higher accuracy on arxiv documents. You also mention that marker will convert fewer equations when compared to nougat, and ye…
Author here: for my use case (converting scientific PDFs in bulk), nougat was the best solution, so I compared to it as the default. I also compare to naive text extraction further down. Nougat is a great model, and converts a lot of PDFs very well. I just wanted something faster, and more generalizable.
I noticed marker downloaded a PyTorch checkpoint called `nougat-0.1.0-small`, do you use nougat under the hood too or is that just a coincidence?