31–40 of 85 posts
How does it do with multi-column text and headers and footers?
Earlier quoted context omitted.
Actually, we have trained the model to convert to markdown and do semantic tagging at the same time. Eg, the equations will be extracted as LaTeX equations, and images (plots, figures, and so on) will be described within the ` ` tags. Same with ` `, ` `, . Also, we extract the tables as HTML tables instead of markdown for complex tables.
What happens to footnotes?
Earlier quoted context omitted.
Hey, the reason for the long processing time is that lots of people are using it, and with probably larger documents. I tested your file locally seems to be working correctly. https://ibb.co/C36RRjYs Regarding the token limit, it depends on the text. We are using the qwen-2.5-vl tokenizer in case you are interested in reading about it. You can run it very easily in a Colab notebook. This should be faster than the dem…
> I tested your file locally seems to be working correctly Apologies if there's some unspoken nuance in this exchange, but by "working correctly" did you just mean that it ran to completion? I don't even recognize some of the unicode characters that it emitted (or maybe you're using some kind of strange font, I guess?) Don't misunderstand me, a ginormous number of floating point numbers attempting to read that handwr…
Log: MA 6100 Z. O 3. 15
34 cement emitter resistors - 0.33 R SW 5% measure 0.29 R, 0.26 R 35 replaced R'4 36, R4 30 emitter resistor on R-44 0.0. 3rd w/ new WW 5W .33R 36 % w/ ceramic lead insulators applied de-oat d100 to Speak outs, card terminals, terminal blocks, output tran jacks 37 replace &-clun diviers and class A BJTs w/ BD139/140 & TIP37A2 38 placed boards back in 39 desoldered ground lus from volume (con 48) 40 contact cleaner, Deox. t DS, facel/42 on pots & switches · teflon lube on rotor joint 41 reably cleaned ground lus & resoldered, reattatched panel ```
You can paste it in https://markdownlivepreview.com/ and see the extraction. This is using the Colab notebook I have shared before.
Which Unicode characters are you mentioning here?
How does it compare to Datalab/Marker https://github.com/datalab-to/marker ? We evaluated many PDF->MD converters and this one performed the best, though it is not perfect.
the inline $\sigma_0$ is mangled as "s 0", and $f(t)$ is mangled as "f~~t*!". The current model gets them both correct.
Earlier quoted context omitted.
Actually, we have trained the model to convert to markdown and do semantic tagging at the same time. Eg, the equations will be extracted as LaTeX equations, and images (plots, figures, and so on) will be described within the ` ` tags. Same with ` `, ` `, . Also, we extract the tables as HTML tables instead of markdown for complex tables.
Have you considered XML. TEI, for example, is very robust and mature for marking up documents.
There are no benchmarks or accuracy measures on a hold out set?
Hi, author of the model here.. We have a benchmark for evaluating VLM on document understanding tasks: https://idp-leaderboard.org/ . But unfortunately, it does not include image to markdown as a task. The problem with evaluating an image to markdown is that even if the order of two blocks are different, it can still be correct. Eg: if you have both seller info and buyer info side by side in the image one model can e…
Earlier quoted context omitted.
Have you considered XML. TEI, for example, is very robust and mature for marking up documents.
First I heard of it. https://en.wikipedia.org/wiki/Text_Encoding_Initiative
How does it do with multi-column text and headers and footers?
We have trained the model on tables with hierarchical column headers and with rowspan and colspan >1. So it should work fine. This is the reason we predict the table in HTML instead of markdown.
There are no benchmarks or accuracy measures on a hold out set?
Hi, author of the model here.. We have a benchmark for evaluating VLM on document understanding tasks: https://idp-leaderboard.org/ . But unfortunately, it does not include image to markdown as a task. The problem with evaluating an image to markdown is that even if the order of two blocks are different, it can still be correct. Eg: if you have both seller info and buyer info side by side in the image one model can e…
OCR that has lower accuracy, but where the inaccurate parts are left blank or flagged are far superior. Mistral OCR also suffers from this problem.
If your OCR produced bounding boxes for every text line, and ran a traditional OCR on the text, this could alleviate it. Or at the very least bounding boxes let users cross-correlate with output from traditional OCR.
Also a small note, it's probably best not to say your product beats Mistral when it's not even tested against it. Having more features doesn't make a product better if the accuracy is not better on those features.
I don't mean to be discouraging, this is an important space and it looks like you have a very feature rich model. I'd like to see a good solution be developed!